Logstash docker with kafka input not consuming data

Elastic Stack Logstash
1

Hello,

I am having problems to make the Logstash docker working as consumer with a kafka input. I can see the tcp socket in the zookeeper node but
I can't get any data in the consumer.

When I look the consumer (offset -1 ?):
Partition LogSize Consumer Offset Lag Consumer Instance Owner
0 136314 -1 136315 logstash_docker_06148a3f844e-1472116989973-2eea36d3-0

I've just one topic in one partition with 3 kafka/zookeeper cluster nodes.
Thats is my logstash conf:
input {

kafka {
    zk_connect => "10.10.20.31:2181"
    topic_id => "topic-events"
    add_field => [ "[@metadata][external-event]", "true"]
    reset_beginning => true
    rebalance_max_retries => 3
    rebalance_backoff_ms => 3000
    consumer_timeout_ms => 30000
}

}
output {
stdout {
codec => "json"
}

And this is my docker info ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
06148a3f844e logstash:2.3.4 "/docker-entrypoint.s" 14 minutes ago Up 13 minutes 0.0.0.0:5043->5043/tcp

I can make telnet to the zookeeper from the docker server and I can "see traffic" using tcpdump.

My Versions running:
Logstash:2.3.4
kafka-input plugin: 2.0.8
Kafka/Zookeeper running: 0.8.2.2 (kafka_2.10-0.8.2.2)

Any ideas?
Thanks very much

2

when I run logstash in verbose mode I got the following:

{:timestamp=>"2016-08-25T13:22:25.646000+0000", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"input", :name=>"kafka", :path=>"logstash/inputs/kafka", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2016-08-25T13:22:26.050000+0000", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"json", :path=>"logstash/codecs/json", :level=>:debug, :file=>"logstash/plugin.rb", :line=>"76", :method=>"lookup"}

{:timestamp=>"2016-08-25T13:22:26.123000+0000", :message=>"Pipeline main started", :file=>"logstash/agent.rb", :line=>"473", :method=>"start_pipeline"}
{:timestamp=>"2016-08-25T13:22:31.127000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:22:36.128000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:22:41.129000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:22:46.129000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:22:51.130000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:22:56.131000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:23:01.133000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:23:06.132000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"flush"}
{:timestamp=>"2016-08-25T13:23:11.132000+0000", :message=>"Pushing flush onto pipeline", :level=>:debug, :file=>"logstash/pipeline.rb", :line=>"458", :method=>"fl

But my plugin is installed...

3

Seems like it doesn't see the plugins at all. I'd reinstall logstash.

Try doing a verbose listing of the plugins: https://www.elastic.co/guide/en/logstash/current/working-with-plugins.html

4

Hi Joe,

Thanks for your reply. Its a docker container and I've tried different versions;

a015f1731584:/# logstash-plugin list --verbose | grep kafka
Ignoring ffi-1.9.13 because its extensions are not built. Try: gem pristine ffi --version 1.9.13
logstash-input-kafka (2.0.8)
logstash-output-kafka (2.0.5)

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a015f1731584 logstash:2.3.4 "/docker-entrypoint.s" About an hour ago Up 4 minutes 0.0.0.0:5043->5043/tcp LogstashNode02

5

Seems like it may be related to this:

and