Logstash input kafka not reading messages from topic

Elastic Stack Logstash
1

I am using kafka as message broker and writing and reading messages using logstash-6.3.1.
Attached document shows all the configuration.
I have producer pipeline, which creates messages in kafka
and consumr pipeline which read messages from kafka. However messages are getting created in kafka topic, but consumer instance of logstash not able to read those messages.
There is no error. Both pipelines running under same instyance of logstash.

Producer pipeline:
100-producer-input.conf
image

200-produccer-output.conf
image

Above pipeline working fine and creating messages to kafka topic.

When I add another pipeline for consumer. it doesnt read messages.
Consumer configuration
100-kafka-producer.conf
image
200-kafka-consumer.conf
image

In the initialization logs when logstash starts, it shows client.id as cbgm-client-0 in the logs. WHy does it append zero to client id?
Whats wrong in my configuration?

thanks
Sunil.

2

One update.
When I dont mention client_id in the configuration of consumer. it shows client.id = logstash-0 in the logs.

image
image.png1869×449 21.6 KB

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.