# Can't get event when using Kafka with logstash **URL:** \ **Category:** Logstash\ **Created:** [April 18, 2017, 2:01pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728 "2017-04-18T14:01:19Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [April 18, 2017, 2:01pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728/1 "2017-04-18T14:01:19Z")
Hi, i'm using ES, LS, Kibana 5.3, beat 5.4 and Kafka 0.10.2 I used winlogbeat to send event log to ES Beat -\> Logstash collector -\> Kafka -\> Logstash Processor -\> ElasticSearch This is my config in logstash collector ``` input { beats { port => 5044 } } filter { if [type] == "wineventlog" and [event_id] == 5156 { drop { } } mutate { add_field => {"beatname" => "%{[@metadata][beat]}"} add_field => {"beattype" => "%{[@metadata][type]}"} } } output { kafka { bootstrap_servers => "kafka-01:9092,kafka-02:9093,kafka-03:9094" codec => plain { format => "%{message}" } topic_id => "logstash-log" } } ``` This is my config in logstash processor ``` input { kafka { bootstrap_servers => "kafka-01:9092,kafka-02:9093,kafka-03:9094" consumer_threads => 8 group_id => "logstash" topics => "logstash-log" } } output { elasticsearch { hosts => ["https://ES-02:9200", "https://ES-03:9200", "https://ES-04:9200"] user => "elastic" cacert => "/etc/logstash/ca.pem" ssl => true password => "mypassword" manage_template => false flush_size => 5000 index => "%{beatname}-%{+xxxx.ww}" document_type => "%{beattype}" } } ``` im using[Kafka Ui tool](https://github.com/yahoo/kafka-manager) for monitor multi-broker cluster I checked in tool, no new message in Kafka, no consumer .... After that i checkd in /var/log/logstash/logstash-plain.log, .. no error. I try to set output is file, got many event log .. So what happen? Something wrong with my logstash config. Thansk ---
**Author:** ![luciferdude](https://avatars.discourse-cdn.com/v4/letter/l/f19dbf/32.png) [@luciferdude](https://discuss.elastic.co/u/luciferdude)\ **Post date:** [April 18, 2017, 5:36pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728/2 "2017-04-18T17:36:57Z")
Hey, I've been having the same issue. I was told that 0.10.1.x is not compatible with LS 5.x . So I installed 6.2.7 version of logstash Plugin for both Input and Output and still it doesn't work for me. I've tried everything. Logstash -\> Kafka -\> ELK Follow my thread, may be it will be useful to troubleshoot your issue. > [@Logstash not forwarding logs to ES via Kafka](https://discuss.elastic.co/t/logstash-not-forwarding-logs-to-es-via-kafka/82602): > > I'm using ELK 5.0.1 and Kafka 0.10.1.0 . I'm not sure why my logs aren't forwarding I installed Kafkacat and was successfully able to Produce and Consume logs from all the 3 servers where Kafka cluster is installed. shipper.conf input { file { start\_position =\> "beginning" path =\> "/var/log/logstash/logstash-plain.log" } } output { kafka { topic\_id =\> "stash" bootstrap\_servers =\> "\:9092,\:9092,\:9092" } } receiver.conf i… ---
**Author:** ![luciferdude](https://avatars.discourse-cdn.com/v4/letter/l/f19dbf/32.png) [@luciferdude](https://discuss.elastic.co/u/luciferdude)\ **Post date:** [April 18, 2017, 10:29pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728/3 "2017-04-18T22:29:37Z")
were you able to get this working?. I'm curious as it can help me solving my issue. ---
**Author:** ![tatdat](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tatdat/32/113160_2.png) [@tatdat](https://discuss.elastic.co/u/tatdat)\ **Post date:** [April 19, 2017, 4:21am UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728/5 "2017-04-19T04:21:59Z")
I read some thread in this forum, they said some version of kafka plugin not support 0.10.1 or 0.10.2 ... wait developer fix it ... Now i used rabbitmq . It's simple and very easy config. If u need help with rabbitmq, i can help u 🙂 ---
**Author:** ![luciferdude](https://avatars.discourse-cdn.com/v4/letter/l/f19dbf/32.png) [@luciferdude](https://discuss.elastic.co/u/luciferdude)\ **Post date:** [April 19, 2017, 5:09pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728/6 "2017-04-19T17:09:41Z")
Thank you. I will look into rabbitmq as well. Appreciate your response. ---
**Author:** ![luciferdude](https://avatars.discourse-cdn.com/v4/letter/l/f19dbf/32.png) [@luciferdude](https://discuss.elastic.co/u/luciferdude)\ **Post date:** [April 19, 2017, 7:51pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728/7 "2017-04-19T19:51:00Z")
Hey , I got mine working. see below my application stack. Also, make sure that port 9092 Ingress is open on your Kafka Cluster from ELK servers. I switched my logging level to 'debug' which helped figure out the issue. Kafka - 2.11\_0.10.1 Zookeeper - 3.4.9-1 ES - 5.0.1 Kibana - 5.0.1 LS - 5.1.1 logstash-input-kafka - 6.2.7 ​​​​​​​logstash-output-kafka - 5.1.1 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 17, 2017, 7:53pm UTC](https://discuss.elastic.co/t/cant-get-event-when-using-kafka-with-logstash/82728/8 "2017-05-17T19:53:57Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.