# Logstash Kafka Input Plugin consumer not reading messages of Kafka topic
**URL:** https://discuss.elastic.co/t/logstash-kafka-input-plugin-consumer-not-reading-messages-of-kafka-topic/91686
**Category:** Logstash
**Created:** [July 3, 2017, 8:07pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-consumer-not-reading-messages-of-kafka-topic/91686 "2017-07-03T20:07:27Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@sgunnala](https://discuss.elastic.co/u/sgunnala)
#### Post date: [July 3, 2017, 8:07pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-consumer-not-reading-messages-of-kafka-topic/91686/1 "2017-07-03T20:07:27Z")
Kafka 0.9 filebeats 5.1 LogstashIndexer: 5.0.2 with Kafka Input plugin
log process generated a 40 mil lag and consumer stopped consuming. Tried restarts and config changes but No luck.
Not sure when it stopped processing but want to consume all those messages. Any help is really appreciated.
Kafka input plugin Conf
input {
kafka {
bootstrap\_servers =\> "url"
topics =\> ["logstash\_topic"]
client\_id =\> " "
group\_id =\> "groupid"
consumer\_threads =\> 1
auto\_offset\_reset =\> "earliest"
enable\_auto\_commit =\> "true"
type =\> "access\_core"
session\_timeout\_ms =\> "30000"
heartbeat\_interval\_ms =\> "1000"
auto\_commit\_interval\_ms =\> "5000"
poll\_timeout\_ms =\> "600"
codec =\> json
}
info logs
[ERROR][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Error ILLEGAL\_GENERATION occurred while committing offsets for group logstash\_topic
[WARN][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Auto offset commit failed: Commit cannot be completed due to group rebalance
[ERROR][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Error ILLEGAL\_GENERATION occurred while committing offsets for group logstash\_topic
[WARN][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator] Auto offset commit failed:
[INFO][org.apache.kafka.clients.consumer.internals.AbstractCoordinator] Marking the coordinator 2147483583 dead.
---
### Author:  [@guyboertje](https://discuss.elastic.co/u/guyboertje)
#### Post date: [July 4, 2017, 1:33pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-consumer-not-reading-messages-of-kafka-topic/91686/2 "2017-07-04T13:33:17Z")
Seems as if you are experiencing this: [comment on Kafka JIRA issue](https://issues.apache.org/jira/browse/KAFKA-2985?focusedCommentId=15106684&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15106684)
My interpretation of this: your LS filter/ouputs stall. This causes backpressure meaning that the LS kafka input threads are blocked waiting to push events into the LS queue and while they are doing this they are not polling.
Use the LS monitoring API to verify that the filters/outputs have not stalled.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 1, 2017, 1:33pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-plugin-consumer-not-reading-messages-of-kafka-topic/91686/3 "2017-08-01T13:33:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.