Does Logstash 2.1.1 work well with Kafka 0.9.0?


(Kenny Qiao) #1

I plan to upgrade Kafka to 0.9.0 and wondering if logstash 2.1.1 could work well with Kafka 0.9.0.
Does anyone have experience using logstash 2.1.1 with Kafka0.9.0 in production?

Thanks.


(Mark Walkom) #2

It's not ready as of this moment - https://github.com/logstash-plugins/logstash-input-kafka/issues/48


(Mike Sparr) #3

Works fine here and pushing soon to production.

Beats -> Logstash -> Kafka <- Logstash -> ES

I'm using it in staging and consumed 150 million events from Beats (top, file, packet) in past few days without issue, published to Kafka topics, and another Logstash instance consumed topics and published to ES. Upgraded ES and LS to 2.1.1, added Beats, and Confluent Platform 2.0 (with Kafka 0.9.0). No issues.



(Joe Lawson) #4

Thanks for the feedback!


(Allen Chan) #5

@mikesparr, Can you share in general your hardware specs?


(system) #6