Input plugin for kafka

(Yuichiro Munakata) #1

Does anyone use logstash input plugin for Kafka?

I try to consume data in Kafka with Kafka input plugin. logstash -debug prints it seems that logstash.conf has been loaded and logstash starts properly. After that logstash starts and shows

15:38:36.156 [[main]-pipeline-manager] INFO logstash.pipeline - Pipeline main started
15:38:36.252 [Api Webserver] DEBUG logstash.agent - Starting puma
15:38:36.258 [Api Webserver] DEBUG logstash.agent - Trying to start WebServer {:port=>9600}
15:38:36.270 [Api Webserver] DEBUG logstash.api.service - [api-service] start
15:38:36.379 [Api Webserver] INFO logstash.agent - Successfully started Logstash API endpoint {:port=>9600}
15:38:41.237 [Ruby-0-Thread-8: /usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:532] DEBUG logstash.pipeline - Pushing flush onto pipeline

I cannot view the data over my Kibana.

Here are my setup.

a. Kafka
b. logstash 5.4.0-1
c. logstash.conf (it's minimum to narrow down the issue)

input {
kafka {
topics => "ockafka"
#enable_auto_commit => "false"

output {
elasticsearch {

Does anyone have a tips to use kafka input plugin? It would be great to share it with me.

Thanks in advance,

(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.