Logstash kafka input plugin:can not read data from kafka


#1

Now,my logstash can not consume data from kafka;
logstash configuration:
input { kafka { bootstrap_servers => group_id => } } filter{ XXXX } output{ elasticsearch{XXX} }

when i see logstash comsuer offset from kafka, i found
Group Topic Pid Offset logSize Lag Owner
logstash log-xxx 0 544167 3598995 3054828 none

and the offset can not be changed,

So i add the configuation: enable_auto_commit=false ; then logstash can read data?
why??? maybe zookeeper can not working well????

anyone help???


(Archelle Pagapulan) #2
  1. Check if kafka broker is running.
  2. Make sure you have the ff config in your input
    kafka {
    topics => ["mytopic"]
    bootstrap_servers => "host1:port,host2:port"
    }

#3

yes,kafka broker is running; and by kafka console consumer,we can get data;

and i have correct config about logstash

input{
   kafka{
        group_id=>
        topics=>
        bootstrap_servers=>
  }
}

(system) #4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.