# Filebeat loses connection to Kafka

**URL:** <https://discuss.elastic.co/t/filebeat-loses-connection-to-kafka/47660>\
**Category:** Beats\
**Created:** [April 18, 2016, 1:37pm UTC](https://discuss.elastic.co/t/filebeat-loses-connection-to-kafka/47660 "2016-04-18T13:37:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![gossu](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gossu](https://discuss.elastic.co/u/gossu)\
**Post date:** [April 18, 2016, 1:37pm UTC](https://discuss.elastic.co/t/filebeat-loses-connection-to-kafka/47660/1 "2016-04-18T13:37:30Z")

</div>

I am using the latest nightly build of Filebeat to output logs from _/var/log/messages_ to Kafka (single node).

The whole thing is working fine for about 30mins and after that Filebeat suddenly stops being able to communicate with Kafka.

Restarting the filebeat service then fixes the issue.

Below the logs I'm getting at the moment when the problem occurs:

```
2016-04-18T13:20:14+03:00 DBG output worker: publish 2 events
2016-04-18T13:20:14+03:00 DBG guaranteed flag is set
2016-04-18T13:20:14+03:00 DBG publish events with attempts=-1
2016-04-18T13:20:14+03:00 DBG forwards msg with attempts=-1
2016-04-18T13:20:14+03:00 DBG message forwarded
2016-04-18T13:20:14+03:00 DBG events from worker worker queue
2016-04-18T13:20:14+03:00 DBG publish events
2016-04-18T13:20:14+03:00 WARN producer/broker/[1 859530387616] state change to [closing] because %!s(MISSING)

2016-04-18T13:20:14+03:00 WARN Closed connection to broker [172.24.33.7:9092]

2016-04-18T13:20:14+03:00 DBG Kafka publish failed with: EOF
2016-04-18T13:20:14+03:00 DBG Kafka publish failed with: EOF
2016-04-18T13:20:14+03:00 DBG finished kafka batch
2016-04-18T13:20:14+03:00 DBG handlePublishEventsResult
2016-04-18T13:20:14+03:00 DBG handle publish error: EOF
2016-04-18T13:20:14+03:00 INFO Error publishing events (retrying): EOF

```

After that I'm getting infinite number of repeated logs looking like that:

```
2016-04-18T13:20:14+03:00 DBG forwards msg with attempts=-1
2016-04-18T13:20:14+03:00 DBG message forwarded
2016-04-18T13:20:14+03:00 DBG events from retries queue
2016-04-18T13:20:14+03:00 DBG publish events
2016-04-18T13:20:14+03:00 DBG Kafka publish failed with: EOF
2016-04-18T13:20:14+03:00 DBG Kafka publish failed with: EOF
2016-04-18T13:20:14+03:00 DBG finished kafka batch
2016-04-18T13:20:14+03:00 DBG handlePublishEventsResult
2016-04-18T13:20:14+03:00 DBG handle publish error: EOF
2016-04-18T13:20:14+03:00 INFO Error publishing events (retrying): EOF

```

Could that be a bug in filebeat?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 18, 2016, 2:38pm UTC](https://discuss.elastic.co/t/filebeat-loses-connection-to-kafka/47660/2 "2016-04-18T14:38:44Z")

</div>

looks like a bug kafka client not reconnecting.

---

<div class="post-metadata">

**Author:** ![gossu](https://avatars.discourse-cdn.com/v4/letter/g/e9bcb4/32.png) [@gossu](https://discuss.elastic.co/u/gossu)\
**Post date:** [April 19, 2016, 8:01am UTC](https://discuss.elastic.co/t/filebeat-loses-connection-to-kafka/47660/3 "2016-04-19T08:01:57Z")

</div>

Should I therefore create an issue here: [https://github.com/elastic/beats/issues](https://github.com/elastic/beats/issues) ?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 19, 2016, 12:41pm UTC](https://discuss.elastic.co/t/filebeat-loses-connection-to-kafka/47660/4 "2016-04-19T12:41:24Z")

</div>

Yes please, I didn't find a ticket. Thanks.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:53pm UTC](https://discuss.elastic.co/t/filebeat-loses-connection-to-kafka/47660/5 "2017-07-05T21:53:01Z")

</div>