Kafka partition keep rebalancing among the logstash instances


(shjdwxy) #1

I use logstash to consume logs from kafka and insert them to es. The topic has 80 partitions so I have 80 logstash instances to consume the topic. Kafka input of each logstash is configured with one thread and all of logstash instances share the same group id.
Occasionally I met the problem that "kafka partition keep rebalancing among the logstash instances". When this happened, I have to reduce the num of logstash and reincrease it later. Does any one meet the same problem?


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.