Does anyone use logstash input plugin for Kafka?
I try to consume data in Kafka with Kafka input plugin. logstash -debug prints it seems that logstash.conf has been loaded and logstash starts properly. After that logstash starts and shows
15:38:36.156 [[main]-pipeline-manager] INFO logstash.pipeline - Pipeline main started
15:38:36.252 [Api Webserver] DEBUG logstash.agent - Starting puma
15:38:36.258 [Api Webserver] DEBUG logstash.agent - Trying to start WebServer {:port=>9600}
15:38:36.270 [Api Webserver] DEBUG logstash.api.service - [api-service] start
15:38:36.379 [Api Webserver] INFO logstash.agent - Successfully started Logstash API endpoint {:port=>9600}
15:38:41.237 [Ruby-0-Thread-8: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline
I cannot view the data over my Kibana.
Here are my setup.
a. Kafka 0.10.0.1
b. logstash 5.4.0-1
c. logstash.conf (it's minimum to narrow down the issue)
input {
kafka {
topics => "ockafka"
#enable_auto_commit => "false"
}
}
output {
elasticsearch {
}
}
Does anyone have a tips to use kafka input plugin? It would be great to share it with me.
Thanks in advance,
Yuichiro