Multiple logstash instances consume log messages in kafka, but only one of them is working. When I close one of them, the other starts to work. How to make two logstash instances work together
You meant only one Logstash instance is actively consuming from Kafka topic?
How many topics and partitions do you have?
How many consumer threads have you configured in kafka input?
If you have 1 topic and 2 partitions, you should set consumer_threads in each Logstash instance to be 1. This will allow both Logstash instances to consume from the same topic concurrently from different partitions.
i.e. the total number of consumer_threads across your Logstash instances (in the same client group) should be equal to the number of partitions you are consuming from.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.