Auto offset commit failed for group logstash


#1

Hi,

I have a set-up with Kafka and ELK. In the logstash side, I frequently receive the following warning message.

23:41:29.767 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator - Marking the coordinator x.x.x.x:x (id: 2147483646 rack: null) dead for group logstash
23:41:29.770 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator - Discovered coordinator x.x.x.x:x (id: 2147483646 rack: null) for group logstash.
23:41:29.896 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] WARN  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto offset commit failed for group logstash: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.
...
23:41:29.990 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] INFO  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Revoking previously assigned partitions [test-0] for group logstash
23:41:29.990 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator - (Re-)joining group logstash
23:41:29.993 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator - (Re-)joining group logstash
23:41:33.300 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] INFO  org.apache.kafka.clients.consumer.internals.AbstractCoordinator - Successfully joined group logstash with generation 647
23:41:33.300 [Ruby-0-Thread-7: /usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-input-kafka-5.1.0/lib/logstash/inputs/kafka.rb:225] INFO  org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Setting newly assigned partitions [test-0] for group logstash

For the first two lines, I guess that kafka considers my logstash client does not respond in a time period, so it disconnects. But reconnect again. For the 3rd line, it is a batch of same warning message. I have followed it, setting session_timeout_ms to "25000" and set max_poll_records to "1". But I still got same above warning message pattern in logstash. Is there any setting I missed? Thanks.

My logstash input
input {
kafka {
topics => ["test"]
bootstrap_servers => "${KAFKA_SERVERS}"
type => 'kafka-input'
max_poll_records => '1'
session_timeout_ms => "25000"
}
}

Michael


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.