# Not able to consume input from a kafka-topic

**URL:** https://discuss.elastic.co/t/not-able-to-consume-input-from-a-kafka-topic/58202
**Category:** Logstash
**Created:** [August 17, 2016, 12:28am UTC](https://discuss.elastic.co/t/not-able-to-consume-input-from-a-kafka-topic/58202 "2016-08-17T00:28:24Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![sokio](https://avatars.discourse-cdn.com/v4/letter/s/e68b1a/32.png) [@sokio](https://discuss.elastic.co/u/sokio)
#### Post date: [August 17, 2016, 12:28am UTC](https://discuss.elastic.co/t/not-able-to-consume-input-from-a-kafka-topic/58202/1 "2016-08-17T00:28:24Z")

</div>

Hi, I am running logstash 2.3.4. I have kafka- 0.9 as the broker.

I also installed the logstash-kafka-input plugin using the matrix given on the github. `0.9 0.9 2.0.0 - 2.3.x 3.x.x`  
using the following command:  
`/opt/logstash/bin/logstash-plugin install --version 3.0.3 logstash-input-kafka`

my input.conf looks like this:

input {  
kafka {  
bootstrap\_servers =\> 'localhost:9092'  
topics =\> 'logstash\_2'  
}  
}

## output { elasticsearch { hosts =\> ["localhost"] } }

on running `/opt/logstash/bin/logstash -f kafka-conf.conf --verbose` I see no error but unfortunately I can not see anytopic on my elasticsearch/kibana web-interface.

Any pointers will be greatly helpful.

---

<div class="post-metadata">

### Author: ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)
#### Post date: [August 17, 2016, 1:13am UTC](https://discuss.elastic.co/t/not-able-to-consume-input-from-a-kafka-topic/58202/2 "2016-08-17T01:13:03Z")

</div>

Are you putting more data into the topic? The auto\_offset\_reset is mandatory. Try `auto_offset_reset => "earliest"`.

Also check your offsets to confirm how the group is behaving. [http://kafka.apache.org/090/documentation.html#basic\_ops\_consumer\_lag](http://kafka.apache.org/090/documentation.html#basic_ops_consumer_lag)

---

<div class="post-metadata">

### Author: ![sokio](https://avatars.discourse-cdn.com/v4/letter/s/e68b1a/32.png) [@sokio](https://discuss.elastic.co/u/sokio)
#### Post date: [August 17, 2016, 4:31pm UTC](https://discuss.elastic.co/t/not-able-to-consume-input-from-a-kafka-topic/58202/3 "2016-08-17T16:31:51Z")

</div>

`auto_offset_reset => "earliest"` works as expected .

My detailed config:

1. logstash-2.3.4 (downloaded with all plugins bundled)
2. kibana-4.5.4
3. elasticsearch-2.3.5
4. kafka 0.9

As long as consumer-group is concerned I expected "logstash" to be the output of consumer groups, but it was throwing error on zookeeper-end. Including explicit `group_id` made it functional.

```
input {
     kafka {
             zk_connect => 'localhost:2181'
             topic_id => 'logstash_topic'
             group_id => "kafka_consumer"
             auto_offset_reset => "smallest"
     }
 }

 output {
     elasticsearch {
                 hosts => ["localhost"]
             }
 }

```

* * *

**Another confusion, how can I check the installed version of a plugin in logstash-plugin list ?**

`bin/logstash-plugin list --verbose`

---

<div class="post-metadata">

### Author: ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)
#### Post date: [August 22, 2016, 3:38pm UTC](https://discuss.elastic.co/t/not-able-to-consume-input-from-a-kafka-topic/58202/4 "2016-08-22T15:38:15Z")

</div>

> [@sokio](#):
>
> Another confusion, how can I check the installed version of a plugin in logstash-plugin list ?
> 
> bin/logstash-plugin list --verbose

That appears correct according to example 2 here: [Working with plugins | Logstash Reference [8.11] | Elastic](https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html#listing-plugins)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:42am UTC](https://discuss.elastic.co/t/not-able-to-consume-input-from-a-kafka-topic/58202/5 "2017-07-06T04:42:16Z")

</div>


