Logstash kafka input consumer group not sharing partitions

[Maxwell_Flanders]

I have a really weird case where I have a 6 logstash consumers consuming from a kafka topic using the logstash-kafka-input plugin. The 6 logstash consumers are their own logstash instances on different machines (and have run with no issue for some time). They have the SAME consumer group, and the topic has 8 partitions.

In the past when I started up a group of logstash instances like this, I could watch the logs as they shifted around and each got assigned a partition or two. However, currently, when I restart these machines I am seeing behavior where the LATEST logstash instance that starts up with take over ALL of the available partitions, stealing them from the last instance that was using them.

this is seriously damaging our ability to consume from kafka using logstash. any thoughts on possible causes would be appreciated!

Using logstash 5.2.

Thank you!

[Maxwell_Flanders]

Update: debug logs show that as soon as a logstash instance connects to kafka, it is assigned all partitions, but then it looks like immediately after that, the consumer dies, with the logs:

[2017-03-07T18:02:28,396][DEBUG][org.apache.kafka.clients.consumer.KafkaConsumer] The Kafka consumer has closed.
[2017-03-07T18:02:28,409][DEBUG][logstash.inputs.kafka    ] closing {:plugin=>"LogStash::Inputs::Kafka"}

There are no other error logs thrown indicating why it is dying though.

[Maxwell_Flanders]

We found the issue, there was a message on the topic of size > max_partition_fetch_bytes. This was killing our consumers as soon as they tried to start up. Each of the consumers would take all partitions and then die in turn.