# Error Outputting data to kafka **URL:** \ **Category:** Logstash\ **Created:** [March 2, 2016, 5:41am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192 "2016-03-02T05:41:52Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![naiktej13](https://avatars.discourse-cdn.com/v4/letter/n/e5b9ba/32.png) [@naiktej13](https://discuss.elastic.co/u/naiktej13)\ **Post date:** [March 2, 2016, 5:41am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/1 "2016-03-02T05:41:52Z")
I want to publish my script data to kafka cluster using logstash. For that I configured logstash on centOS hear is the config file i wrote: ``` input { exec { command => "python /home/elastic/test.py" interval => 6 codec => "json_lines" } } output { kafka { codec => plain { format => "%{message}"} topic_id => "logstash" bootstrap_servers => "kafka-1:2181,kafka-2:2181,kafka-3:2181" } stdout{ codec => "rubydebug" } } ``` **My kafka cluster is running elsewhere and this forwarding logstash machine does not have any kafka components** When I try to run this file I get following error: > log4j, [TS] WARN: org.apache.kafka.common.network.Selector: Error in I/O with kafka-1/xx.xx.xx.xx > java.net.ConnectException: Connection refused > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.finishConnect(Unknown Source) > at org.apache.kafka.common.network.Selector.poll(Selector.java:238) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:192) > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:191) > at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:122) > at java.lang.Thread.run(Unknown Source) Please Suggest. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 2, 2016, 5:46am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/2 "2016-03-02T05:46:37Z")
Can you telnet from the LS server to the kafka one? What version of things are you running? ---
**Author:** ![naiktej13](https://avatars.discourse-cdn.com/v4/letter/n/e5b9ba/32.png) [@naiktej13](https://discuss.elastic.co/u/naiktej13)\ **Post date:** [March 2, 2016, 6:01am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/3 "2016-03-02T06:01:28Z")
thanks @warkolm My current configuration is : Logstash: 2.2.0 Kafka: 0.9.0.1 OS: CentOS/RHEL 7 on every machine > [root@zx logstash-2.2.0]# nmap -v kafka-2 -p 2181 > Starting Nmap 6.40 ( [http://nmap.org](http://nmap.org) ) at 2016-03-02 00:57 EST > Initiating ARP Ping Scan at 00:57 > Scanning kafka-2 [1 port] > Completed ARP Ping Scan at 00:57, 0.01s elapsed (1 total hosts) > Initiating SYN Stealth Scan at 00:57 > Scanning kafka-2 [1 port] > Discovered open port 2181/tcp on xx.xx.xx.xx > Completed SYN Stealth Scan at 00:57, 0.01s elapsed (1 total ports) > Nmap scan report for kafka-2 ( xx.xx.xx.xx) > Host is up (0.00031s latency). > PORT STATE SERVICE > 2181/tcp open unknown ---
**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\ **Post date:** [March 2, 2016, 6:28pm UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/4 "2016-03-02T18:28:57Z")
Most likely the bootstrap servers are returning a different host name than you expect. Make sure your that your broker server.properties has [host.name](http://host.name) set to what the producers should use as that is the value returned via the bootstrap metadata. See [http://stackoverflow.com/a/20514389](http://stackoverflow.com/a/20514389) ---
**Author:** ![naiktej13](https://avatars.discourse-cdn.com/v4/letter/n/e5b9ba/32.png) [@naiktej13](https://discuss.elastic.co/u/naiktej13)\ **Post date:** [March 3, 2016, 5:26am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/5 "2016-03-03T05:26:09Z")
My server.properties snippet is as follow: ``` ... # The port the socket server listens on port=9092 # Hostname the broker will bind to. If not set, the server will bind to all interfaces host.name=kafka-1 # Hostname the broker will advertise to producers and consumers. If not set, it uses the # value for "host.name" if configured. Otherwise, it will use the value returned from # java.net.InetAddress.getCanonicalHostName(). advertised.host.name=kafka-1 ... ``` All producers can ping the brokers using their hostname and the following lines are added in **_/etc/hosts_** file ``` ... xx.xx.xx.xx kafka-2 xx.xx.xx.xx kafka-3 xx.xx.xx.xx kafka-1 ... ``` I still can't figure out where the error lies? ---
**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\ **Post date:** [March 3, 2016, 5:53pm UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/6 "2016-03-03T17:53:05Z")
Are you able to run the kafka console producer from the host giving you trouble? The error implies that it couldn't reach the kafka-1 host so I'd at least check for errors when the message is being created on the host and perhaps ZK as well. Also are you seeing errors for all hosts? I'd expect that if it fails on one, it would go to the next one and keep working. There is an issue with Kafka 0.8.2.X that blocks on sends when the broker is down. Try setting [metadata\_fetch\_timeout\_ms](https://www.elastic.co/guide/en/logstash/current/plugins-outputs-kafka.html#plugins-outputs-kafka-metadata_fetch_timeout_ms) lower. To like 500 for your test. ---
**Author:** ![Joe\_Lawson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joe_lawson/32/3390_2.png) [@Joe\_Lawson](https://discuss.elastic.co/u/Joe_Lawson)\ **Post date:** [March 3, 2016, 5:53pm UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/7 "2016-03-03T17:53:39Z")
PS Here is the issue I was talking about: [http://mail-archives.apache.org/mod\_mbox/kafka-users/201507.mbox/\](http://mail-archives.apache.org/mod_mbox/kafka-users/201507.mbox/%3CCAAUywg-je-pzDqVvOWndrRnpDkf42nTN5BB092ZpRiUSiY0vOw@mail.gmail.com%3E) ---
**Author:** ![Srinivas\_Chamarthi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/srinivas_chamarthi/32/8729_2.png) [@Srinivas\_Chamarthi](https://discuss.elastic.co/u/Srinivas_Chamarthi)\ **Post date:** [June 30, 2016, 4:07am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/8 "2016-06-30T04:07:43Z")
Hi, did you figure out the issue ? I ran into same issue and breaking my head for last four hours. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:50am UTC](https://discuss.elastic.co/t/error-outputting-data-to-kafka/43192/9 "2017-07-06T04:50:16Z")