# Logstash docker with kafka input not consuming data **URL:** \ **Category:** Logstash\ **Created:** [August 25, 2016, 9:38am UTC](https://discuss.elastic.co/t/logstash-docker-with-kafka-input-not-consuming-data/58908 "2016-08-25T09:38:02Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![cesarpball](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesarpball/32/11590_2.png) [@cesarpball](https://discuss.elastic.co/u/cesarpball)\ **Post date:** [August 25, 2016, 9:38am UTC](https://discuss.elastic.co/t/logstash-docker-with-kafka-input-not-consuming-data/58908/1 "2016-08-25T09:38:02Z")
Hello, I am having problems to make the Logstash docker working as consumer with a kafka input. I can see the tcp socket in the zookeeper node but I can't get any data in the consumer. When I look the consumer (offset -1 ?): Partition LogSize Consumer Offset Lag Consumer Instance Owner 0 136314 -1 136315 logstash\_docker\_06148a3f844e-1472116989973-2eea36d3-0 I've just one topic in one partition with 3 kafka/zookeeper cluster nodes. Thats is my logstash conf: input { ``` kafka { zk_connect => "10.10.20.31:2181" topic_id => "topic-events" add_field => ["[@metadata][external-event]", "true"] reset_beginning => true rebalance_max_retries => 3 rebalance_backoff_ms => 3000 consumer_timeout_ms => 30000 } ``` } output { stdout { codec =\> "json" } And this is my docker info ps: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 06148a3f844e logstash:2.3.4 "/docker-entrypoint.s" 14 minutes ago Up 13 minutes 0.0.0.0:5043-\>5043/tcp I can make telnet to the zookeeper from the docker server and I can "see traffic" using tcpdump. My Versions running: Logstash:2.3.4 kafka-input plugin: 2.0.8 Kafka/Zookeeper running: 0.8.2.2 (kafka\_2.10-0.8.2.2) Any ideas? Thanks very much ---
**Author:** ![cesarpball](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesarpball/32/11590_2.png) [@cesarpball](https://discuss.elastic.co/u/cesarpball)\ **Post date:** [August 25, 2016, 1:37pm UTC](https://discuss.elastic.co/t/logstash-docker-with-kafka-input-not-consuming-data/58908/2 "2016-08-25T13:37:50Z")
when I run logstash in verbose mode I got the following: {:timestamp=\>"2016-08-25T13:22:25.646000+0000", :message=\>"Plugin not defined in namespace, checking for plugin file", :type=\>"input", :name=\>"kafka", :path=\>"logstash/inputs/kafka", :level=\>:debug, :file=\>"logstash/plugin.rb", :line=\>"76", :method=\>"lookup"} {:timestamp=\>"2016-08-25T13:22:26.050000+0000", :message=\>"Plugin not defined in namespace, checking for plugin file", :type=\>"codec", :name=\>"json", :path=\>"logstash/codecs/json", :level=\>:debug, :file=\>"logstash/plugin.rb", :line=\>"76", :method=\>"lookup"} {:timestamp=\>"2016-08-25T13:22:26.123000+0000", :message=\>"Pipeline main started", :file=\>"logstash/agent.rb", :line=\>"473", :method=\>"start\_pipeline"} {:timestamp=\>"2016-08-25T13:22:31.127000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:22:36.128000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:22:41.129000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:22:46.129000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:22:51.130000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:22:56.131000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:23:01.133000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:23:06.132000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"flush"} {:timestamp=\>"2016-08-25T13:23:11.132000+0000", :message=\>"Pushing flush onto pipeline", :level=\>:debug, :file=\>"logstash/pipeline.rb", :line=\>"458", :method=\>"fl But my plugin is installed... ---
**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\ **Post date:** [August 25, 2016, 2:20pm UTC](https://discuss.elastic.co/t/logstash-docker-with-kafka-input-not-consuming-data/58908/3 "2016-08-25T14:20:07Z")
Seems like it doesn't see the plugins at all. I'd reinstall logstash. Try doing a verbose listing of the plugins: [https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html](https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html) ---
**Author:** ![cesarpball](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesarpball/32/11590_2.png) [@cesarpball](https://discuss.elastic.co/u/cesarpball)\ **Post date:** [August 25, 2016, 2:25pm UTC](https://discuss.elastic.co/t/logstash-docker-with-kafka-input-not-consuming-data/58908/4 "2016-08-25T14:25:04Z")
Hi Joe, Thanks for your reply. Its a docker container and I've tried different versions; a015f1731584:/# logstash-plugin list --verbose | grep kafka Ignoring ffi-1.9.13 because its extensions are not built. Try: gem pristine ffi --version 1.9.13 logstash-input-kafka (2.0.8) logstash-output-kafka (2.0.5) CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES a015f1731584 logstash:2.3.4 "/docker-entrypoint.s" About an hour ago Up 4 minutes 0.0.0.0:5043-\>5043/tcp LogstashNode02 ---
**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\ **Post date:** [August 25, 2016, 2:40pm UTC](https://discuss.elastic.co/t/logstash-docker-with-kafka-input-not-consuming-data/58908/5 "2016-08-25T14:40:22Z")
Seems like it may be related to this: > and > ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:41am UTC](https://discuss.elastic.co/t/logstash-docker-with-kafka-input-not-consuming-data/58908/6 "2017-07-06T04:41:36Z")