Logstash Kafka input loosing events

Hi All,

I use logstash 2.4.1 with logstash-input-kafka=5.1.6 and output to ES 2.4, kafka 0.10, 160 partitions.
I insert 1000 events to a kafka topic and only about half of the events are being written to ES, I don't see any errors.

If I increase the number of consumer_threads to 160 almost all the events are being written to ES, but no all.

In any case even one consumer_group should fetch all the events even if slower.

What could be the problem?


This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.