# Filebeat EOF - And then succeeds at second try

**URL:** <https://discuss.elastic.co/t/filebeat-eof-and-then-succeeds-at-second-try/65015>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 4, 2016, 2:40pm UTC](https://discuss.elastic.co/t/filebeat-eof-and-then-succeeds-at-second-try/65015 "2016-11-04T14:40:05Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Christian\_Norskov](https://avatars.discourse-cdn.com/v4/letter/c/df788c/32.png) [@Christian\_Norskov](https://discuss.elastic.co/u/Christian_Norskov)\
**Post date:** [November 4, 2016, 2:40pm UTC](https://discuss.elastic.co/t/filebeat-eof-and-then-succeeds-at-second-try/65015/1 "2016-11-04T14:40:05Z")

</div>

Hello,

I'm having some issues with filebeats and an error message. (Almost) everytime my filebeat instances tries to send data to logstash, the following logs are written. So it looks like it fails first try to send to logstash, but always succeeds at second try.

> 11/4/2016 2:57:16 PM2016/11/04 13:57:16.671410 single.go:76: INFO Error publishing events (retrying): EOF  
> 11/4/2016 2:57:16 PM2016/11/04 13:57:16.671437 single.go:152: INFO send fail  
> 11/4/2016 2:57:16 PM2016/11/04 13:57:16.671448 single.go:159: INFO backoff retry: 1s  
> 11/4/2016 2:57:17 PM2016/11/04 13:57:17.698679 publish.go:104: INFO Events sent: 5  
> 11/4/2016 2:57:17 PM2016/11/04 13:57:17.698869 registrar.go:163: INFO Registry file updated. 1 states written.

I've set the congestion\_threshold in logstash to 60 seconds, to see if this was the problem, but it didn't help. Neither logstash or filebeat are under any heavy load (about 300kb of logs are created every hour in this test setup).

Notes: Applications are deployed in docker containers in a rancher environment, where filebeat is a sidekick to each application, reading that applications logs. Logstash is also deployed in a container, but kafka is running directly on the hosts (and are accessible to logstash).

Of course this isn't the biggest issue, as logs are succesfully send, but I would rather they were send on first try, so the filebeat logs does not get filled with data, that could potentially overshadow actual problems.

I hope you guys can help.

Cheers,  
Christian

**Configurations**

Logstash config:

> input {  
> beats {  
> port =\> 5044  
> congestion\_threshold =\> 60  
> }  
> }  
> output {  
> kafka {  
> client\_id =\> "client\_id"  
> topic\_id =\> "kafka\_topic"  
> bootstrap\_servers =\> "kafka\_servers"  
> codec =\> plain {  
> format =\> "%{source} =\> %{message}"  
> }  
> }  
> }

Filebeat conig:

> filebeat:
> 
> # List of prospectors to fetch data.
> 
> prospectors:  
> # Each - is a prospector. Below are the prospector specific configurations  
> - paths:  
> - "${LOG\_PATTERN:/var/log/\*\*/\*.log}"  
> output:  
> logstash:  
> hosts: ["${LOGSTASH\_HOST}:${LOGSTASH\_PORT}"]  
> logging:  
> to\_files: true  
> level: INFO  
> files:  
> name: filebeat.log  
> path: /tmp  
> keepfiles: 1

**EDIT**  
Should also note, that there is nothing in the logstash logs besides:

> 11/4/2016 1:38:09 PMSettings: Default pipeline workers: 2  
> 11/4/2016 1:38:09 PMPipeline main started

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [November 7, 2016, 1:46pm UTC](https://discuss.elastic.co/t/filebeat-eof-and-then-succeeds-at-second-try/65015/2 "2016-11-07T13:46:17Z")

</div>

which filebeat/logstash versions are you using? Did you consider upgrading to 5.0 (the `congestion_threshold` is been replaced in newer beats input plugin in logstash).

Anything in logstash logs? EOF means the TCP connection filebeat is using has been closed by some remote.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 25, 2016, 2:40pm UTC](https://discuss.elastic.co/t/filebeat-eof-and-then-succeeds-at-second-try/65015/3 "2016-11-25T14:40:07Z")

</div>

This topic was automatically closed after 21 days. New replies are no longer allowed.