I use logstash to consume logs from kafka and insert them to es. The topic has 80 partitions so I have 80 logstash instances to consume the topic. Kafka input of each logstash is configured with one thread and all of logstash instances share the same group id.
Occasionally I met the problem that "kafka partition keep rebalancing among the logstash instances". When this happened, I have to reduce the num of logstash and reincrease it later. Does any one meet the same problem?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.