Does Logstash 2.1.1 work well with Kafka 0.9.0?

Kenny_Qiao · January 11, 2016, 5:15am

I plan to upgrade Kafka to 0.9.0 and wondering if logstash 2.1.1 could work well with Kafka 0.9.0.
Does anyone have experience using logstash 2.1.1 with Kafka0.9.0 in production?

Thanks.

warkolm · January 11, 2016, 1:22pm

It's not ready as of this moment - https://github.com/logstash-plugins/logstash-input-kafka/issues/48

mikesparr · January 11, 2016, 7:12pm

Works fine here and pushing soon to production.

Beats -> Logstash -> Kafka <- Logstash -> ES

I'm using it in staging and consumed 150 million events from Beats (top, file, packet) in past few days without issue, published to Kafka topics, and another Logstash instance consumed topics and published to ES. Upgraded ES and LS to 2.1.1, added Beats, and Confluent Platform 2.0 (with Kafka 0.9.0). No issues.

Joe_Lawson · January 11, 2016, 9:41pm

Thanks for the feedback!

allenmchan · January 13, 2016, 1:39am

@mikesparr, Can you share in general your hardware specs?

Topic		Replies	Views
Logstash 2.3.1 kafka plugins ssl support Logstash	8	2297	July 6, 2017
Kafka 2.4.1 compatiblity with Logstash and Filebeat 7.10.2 Beats filebeat	1	269	June 23, 2021
Filebeat 5.x pushing to Elasticsearch 6.8.x via Kafka topic Beats filebeat	2	296	April 13, 2020
Kafka to Logstash pipeline not working Logstash	3	646	July 6, 2017
Logstash 6.8 or 7x comaptibility with Kafka 0.9 Logstash	1	297	October 11, 2019

Does Logstash 2.1.1 work well with Kafka 0.9.0?

Related topics