# Filebeat ERR Failed to publish events caused by: EOF **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 8, 2017, 7:11pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893 "2017-01-08T19:11:18Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Gude.ravi](https://avatars.discourse-cdn.com/v4/letter/g/73ab20/32.png) [@Gude.ravi](https://discuss.elastic.co/u/Gude.ravi)\ **Post date:** [January 8, 2017, 7:11pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/1 "2017-01-08T19:11:18Z")
The following is my filebeat yml. The error I am seeing is . Filebeat - ERR Failed to publish events caused by: EOF I understand its a connection closing issue by logstash. I do see the data in elasticsearch. what can be changed on logstash for fixing the above error. I am not using ssl either on logstash or filebeat. Or should I just ignore the error since I am getting the data in elasticsearch ? I think a blog on this topic would help. prospectors: - input\_type: log paths: - "/prod/app.log" fields\_under\_root: true fields: app: true api\_name: dtls - input\_type: log paths: - "/prod/perf.log" fields\_under\_root: true fields: perf: true api\_name: dtls registry\_file: .applog-optset output: logstash: hosts: ["[logstash.whatever.com:5041](http://logstash.whatever.com:5041)"] timeout: 30 logging: to\_files: true files: path: /var/log/filebeat/ name: filebeat\_es\_logs.log keepfiles: 7 level: debug The following is the log 2017-01-08T13:50:56-05:00 DBG output worker: publish 44 events 2017-01-08T13:50:56-05:00 DBG Try to publish 44 events to logstash with window size 62 2017-01-08T13:50:56-05:00 DBG handle error: EOF 2017-01-08T13:50:56-05:00 DBG closing 2017-01-08T13:50:56-05:00 DBG 0 events out of 44 events sent to logstash. Continue sending 2017-01-08T13:50:56-05:00 DBG close connection 2017-01-08T13:50:56-05:00 ERR Failed to publish events caused by: EOF 2017-01-08T13:50:56-05:00 INFO Error publishing events (retrying): EOF 2017-01-08T13:50:56-05:00 DBG close connection 2017-01-08T13:50:56-05:00 DBG send fail 2017-01-08T13:50:56-05:00 DBG End of file reached: /prod/perf.log; Backoff now. 2017-01-08T13:50:56-05:00 DBG End of file reached: /prod/app.log; Backoff now. 2017-01-08T13:50:57-05:00 DBG connect 2017-01-08T13:50:57-05:00 DBG Try to publish 44 events to logstash with window size 31 2017-01-08T13:50:57-05:00 DBG update current window size: 31 2017-01-08T13:50:57-05:00 DBG 31 events out of 44 events sent to logstash. Continue sending 2017-01-08T13:50:57-05:00 DBG Try to publish 13 events to logstash with window size 47 2017-01-08T13:50:57-05:00 DBG 13 events out of 13 events sent to logstash. Continue sending 2017-01-08T13:50:57-05:00 DBG send completed ---
**Author:** ![maddin2016](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/maddin2016/32/16599_2.png) [@maddin2016](https://discuss.elastic.co/u/maddin2016)\ **Post date:** [January 8, 2017, 10:10pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/2 "2017-01-08T22:10:01Z")
It first try to send async to logstash with window size `62`. Then it [decrease](https://github.com/elastic/beats/pull/1689/files#diff-974450088011f7181339087adde4bfcdR113) it to 31 and successfully send to logstash. @ruflin, @andrewkroh is there a way to set this value smaller then 64 or is this value computed? ---
**Author:** ![Gude.ravi](https://avatars.discourse-cdn.com/v4/letter/g/73ab20/32.png) [@Gude.ravi](https://discuss.elastic.co/u/Gude.ravi)\ **Post date:** [January 9, 2017, 12:14pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/3 "2017-01-09T12:14:26Z")
Thank you . Is window size configurable ? I thought the window size is dynamic and depends on the traffic between logstash and filebeat. ---
**Author:** ![tudor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tudor/32/3753_2.png) [@tudor](https://discuss.elastic.co/u/tudor)\ **Post date:** [January 9, 2017, 1:35pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/4 "2017-01-09T13:35:14Z")
Do you get any errors on the Logstash side? Logstash, at least in some versions, closes the connection if its internal queue blocks for more than 5 seconds. But there should be logs indicating that this is happening. ---
**Author:** ![Gude.ravi](https://avatars.discourse-cdn.com/v4/letter/g/73ab20/32.png) [@Gude.ravi](https://discuss.elastic.co/u/Gude.ravi)\ **Post date:** [January 9, 2017, 2:16pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/5 "2017-01-09T14:16:33Z")
Yes on logstash side, I do see errors sometimes like closing connection. but so far I didnt see any issues like documents are being dropped. is window size configurable ? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 9, 2017, 7:12pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/6 "2017-01-09T19:12:34Z")
Which logstash version and which beats input plugin version have you installed? Only the maximum window size is configurable. Some logstash versions are prone to close connections if a batch is too big or takes to long to process within logstash itself. This can result in duplicate events. Worse, if same batch is resend over and over again as is with logstash failing, one can overload logstash itself (and the final output) with duplicates, without ever making progress (known issue for original logstash-forwarder). For this reason beats try to adapt the window size up to [bulk\_max\_size](https://www.elastic.co/guide/en/beats/filebeat/current/logstash-output.html#_bulk_max_size_2) (default: 2048). Newer logstash-input-beats plugin has been rewritten (get most recent version) to send intermediate ACKs as keep-alive signal. This way, logstash has not to drop the connection if it gets congested. ---
**Author:** ![Gude.ravi](https://avatars.discourse-cdn.com/v4/letter/g/73ab20/32.png) [@Gude.ravi](https://discuss.elastic.co/u/Gude.ravi)\ **Post date:** [January 9, 2017, 7:51pm UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/7 "2017-01-09T19:51:46Z")
Thanks Steffen. Version I am using is 5.0.1 ( both ). Please let me know if I should migrate to a newer version. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 10, 2017, 11:32am UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/8 "2017-01-10T11:32:50Z")
I guess you didn't update the logstash-input-beats plugin? I'm not sure if logstash 5.1.1 ships with most recent plugin version. From [logstash-input-beats plugin changelog](https://github.com/logstash-plugins/logstash-input-beats/blob/master/CHANGELOG.md) I'd suggest to have at least version 3.1.11 installed. You should be able to upgrade the plugin using the `bin/logstash-plugin` tool shipped with logstash. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 7, 2017, 11:32am UTC](https://discuss.elastic.co/t/filebeat-err-failed-to-publish-events-caused-by-eof/70893/9 "2017-02-07T11:32:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.