# Logstash kafka input issues in reconnecting to zookeeper

**URL:** <https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741>\
**Category:** Logstash\
**Created:** [August 10, 2016, 8:21pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741 "2016-08-10T20:21:51Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![prabhakar349](https://avatars.discourse-cdn.com/v4/letter/p/5e9695/32.png) [@prabhakar349](https://discuss.elastic.co/u/prabhakar349)\
**Post date:** [August 10, 2016, 8:21pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741/1 "2016-08-10T20:21:51Z")

</div>

We are using logstash version : logstash-2.3.3

Right now we are using kafka input to read messages from broker which is version 0.9

```auto

{
 
  kafka {
    topic_id => "{{kafka_log_consumer_topic_id}}"
    group_id => "{{kafka_consumer_group_id}}"
    zk_connect => "{{zk_consumer_connect_url}}"
  }
 
}
 
filter{
if [level] == "ERROR" or [level] == "WARN" or [level] == "FATAL" {
 
mutate{
  add_field => {
    "thrownMessage" => "%{[thrown][message]}"
    "applicationName" => "%{[contextMap][applicationName]}"
    "errorId" => "%{[contextMap][errorId]}"
   }
}
uuid{
   target => "logEventId"
}
}
}
 
output {
 
if [level] == "ERROR" or [level] == "WARN" or [level] == "FATAL" {
 
  elasticsearch {
    hosts => "{{elastic_search_hosts}}"
  }
 
  kafka {
    topic_id => "{{kafka_error_publish_topic}}"
    bootstrap_servers => "{{kafka_error_publish_host}}"
  }
 
}
else {
  elasticsearch {
    hosts => "{{elastic_search_hosts}}"
  }
}
}

```

Sometimes when our zookeeper goes down , and gets back up after sometime logstash is not reconnecting back again.

Does logstash kafka input supports using the new kafka consumer API 's which is not dependent on zookeeper ?

---

<div class="post-metadata">

**Author:** ![prabhakar349](https://avatars.discourse-cdn.com/v4/letter/p/5e9695/32.png) [@prabhakar349](https://discuss.elastic.co/u/prabhakar349)\
**Post date:** [August 15, 2016, 2:26pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741/2 "2016-08-15T14:26:07Z")

</div>

wanted to know in case if we can refresh zookeeper connections in case of zookeeper failure

---

<div class="post-metadata">

**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\
**Post date:** [August 15, 2016, 4:11pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741/3 "2016-08-15T16:11:58Z")

</div>

The input does support the new APIs but you need to use the right plugin version. Check [https://github.com/logstash-plugins/logstash-input-kafka#kafka-compatibility](https://github.com/logstash-plugins/logstash-input-kafka#kafka-compatibility) for reference.

---

<div class="post-metadata">

**Author:** ![prabhakar349](https://avatars.discourse-cdn.com/v4/letter/p/5e9695/32.png) [@prabhakar349](https://discuss.elastic.co/u/prabhakar349)\
**Post date:** [August 15, 2016, 5:18pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741/4 "2016-08-15T17:18:11Z")

</div>

The matrix says I need to use 2.4 of logstash and I and don't find that anywhere , even in git I see only 2.3.X and version 5.0.X but not the 2.4.

0.9 2.4, 5.0 4.0.0 4.0.0

---

<div class="post-metadata">

**Author:** ![prabhakar349](https://avatars.discourse-cdn.com/v4/letter/p/5e9695/32.png) [@prabhakar349](https://discuss.elastic.co/u/prabhakar349)\
**Post date:** [August 15, 2016, 11:22pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741/5 "2016-08-15T23:22:52Z")

</div>

I see there is a 2.4 branch but not sure where to download the same

---

<div class="post-metadata">

**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\
**Post date:** [August 16, 2016, 1:29pm UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741/6 "2016-08-16T13:29:17Z")

</div>

Yeah it is confusing. The plugins are really rubygem packs. So the matrix says for Logstash 2.3.3 you should use 3.0.0 and it looks like the latest logstash-input-kafka 3.0.x is 3.0.3 [https://rubygems.org/gems/logstash-input-kafka/versions/3.0.3](https://rubygems.org/gems/logstash-input-kafka/versions/3.0.3).

so you need to use the [logstash-plugin](https://www.elastic.co/guide/en/logstash/2.3/working-with-plugins.html#installing-plugins) tool and install that version.

`logstash-plugin install --version 3.0.3 logstash-input-kafka`

Those have the `bootstrap_servers` you want: [https://github.com/logstash-plugins/logstash-input-kafka/blob/v3.0.2/lib/logstash/inputs/kafka.rb#L51](https://github.com/logstash-plugins/logstash-input-kafka/blob/v3.0.2/lib/logstash/inputs/kafka.rb#L51)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:43am UTC](https://discuss.elastic.co/t/logstash-kafka-input-issues-in-reconnecting-to-zookeeper/57741/7 "2017-07-06T04:43:19Z")

</div>