Logstash Kafka input loosing events

Elastic Stack Logstash
1

Hi All,

I use logstash 2.4.1 with logstash-input-kafka=5.1.6 and output to ES 2.4, kafka 0.10, 160 partitions.
I insert 1000 events to a kafka topic and only about half of the events are being written to ES, I don't see any errors.

If I increase the number of consumer_threads to 160 almost all the events are being written to ES, but no all.

In any case even one consumer_group should fetch all the events even if slower.

What could be the problem?

Thanks
Guy

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.