# Logstash 2.1 and kafka 0.9

**URL:** <https://discuss.elastic.co/t/logstash-2-1-and-kafka-0-9/35666>\
**Category:** Logstash\
**Created:** [November 26, 2015, 12:26pm UTC](https://discuss.elastic.co/t/logstash-2-1-and-kafka-0-9/35666 "2015-11-26T12:26:40Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [November 26, 2015, 12:26pm UTC](https://discuss.elastic.co/t/logstash-2-1-and-kafka-0-9/35666/1 "2015-11-26T12:26:40Z")

</div>

Hi,  
I'm looking at using kafka as a message queue in our elk environment. I have two logstash instances running, the first takes inbound json and outputs to kafka while the second reads from kafka and outputs to elasticsearch.

Right now I have one message sitting in kafka waiting to be consumed:

$ bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic logstash-logs --time -1  
logstash-logs:0:1

So, the kafka output looks to be working while the kafka input isn't. Here is my input config:

input {

kafka {  
zk\_connect =\> "testbroker:2181"  
topic\_id =\> "logstash-logs"  
tags =\> "kafka-input"  
queue\_size =\> 20  
fetch\_message\_max\_bytes =\> 1048576  
consumer\_threads =\> 1  
consumer\_id =\> "ls-indexer"  
}  
} # end input

I've tested connectivity to the zookeeper port and that looks fine. And I've tested the elasticsearch output successfully via another logstash instance which passes messages straight through. So, as I'm relatively new to logstash, could someone help me with troubleshooting the failure to consume/send to elasticsearch please?

Regards,  
David

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [November 26, 2015, 11:49pm UTC](https://discuss.elastic.co/t/logstash-2-1-and-kafka-0-9/35666/2 "2015-11-26T23:49:46Z")

</div>

See [https://github.com/logstash-plugins/logstash-input-kafka/issues/48](https://github.com/logstash-plugins/logstash-input-kafka/issues/48)  
We don't support Kafka 0.9 as yet unfortunately.

---

<div class="post-metadata">

**Author:** ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro)\
**Post date:** [December 2, 2015, 11:17am UTC](https://discuss.elastic.co/t/logstash-2-1-and-kafka-0-9/35666/3 "2015-12-02T11:17:04Z")

</div>

Ok, as messages were being delivered I presumed there might be backward compatibility. I've now changed brokers to 0.8.2.1

---

<div class="post-metadata">

**Author:** ![Deb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/deb/32/46162_2.png) [@Deb](https://discuss.elastic.co/u/Deb)\
**Post date:** [December 29, 2015, 11:38am UTC](https://discuss.elastic.co/t/logstash-2-1-and-kafka-0-9/35666/4 "2015-12-29T11:38:26Z")

</div>

@warklom - Any update when Kafka 0.9 will be supported?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:17am UTC](https://discuss.elastic.co/t/logstash-2-1-and-kafka-0-9/35666/5 "2017-07-06T05:17:13Z")

</div>