# Filebeat sends logs, but only at the second try **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 27, 2016, 4:09pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082 "2016-12-27T16:09:53Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Gamer](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@Gamer](https://discuss.elastic.co/u/Gamer)\ **Post date:** [December 27, 2016, 4:09pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/1 "2016-12-27T16:09:53Z")
Hello! So. It seems our case's similar to that ([Filebeat EOF - And then succeeds at second try](https://discuss.elastic.co/t/filebeat-eof-and-then-succeeds-at-second-try/65015)). We have errors in filebeat stating that: ``` 2016-12-27T18:25:51+03:00 DBG Try to publish 2 events to logstash with window size 1 2016-12-27T18:25:51+03:00 DBG handle error: EOF 2016-12-27T18:25:51+03:00 DBG closing 2016-12-27T18:25:51+03:00 DBG 0 events out of 2 events sent to logstash. Continue sending 2016-12-27T18:25:51+03:00 DBG close connection 2016-12-27T18:25:51+03:00 ERR Failed to publish events caused by: EOF 2016-12-27T18:25:51+03:00 INFO Error publishing events (retrying): EOF 2016-12-27T18:25:51+03:00 DBG close connection 2016-12-27T18:25:51+03:00 DBG send fail 2016-12-27T18:25:52+03:00 DBG connect 2016-12-27T18:25:52+03:00 DBG Try to publish 2 events to logstash with window size 1 2016-12-27T18:25:52+03:00 DBG 1 events out of 2 events sent to logstash. Continue sending 2016-12-27T18:25:52+03:00 DBG Try to publish 1 events to logstash with window size 1 2016-12-27T18:25:52+03:00 DBG 1 events out of 1 events sent to logstash. Continue sending 2016-12-27T18:25:52+03:00 DBG send completed 2016-12-27T18:25:52+03:00 DBG Events sent: 2 2016-12-27T18:25:52+03:00 DBG Processing 2 events 2016-12-27T18:25:52+03:00 DBG Registrar states cleaned up. Before: 20, After: 20 2016-12-27T18:25:52+03:00 DBG Write registry file: /var/lib/filebeat/registry ``` In other words logs'are sent, but after failing once. And there's nothing in Logstash logs. Filebeat config (actually there're many more prospectors, but the differences only in the path and quantity of the prospectors): ``` ############################# Filebeat ###################################### filebeat: prospectors: - paths: - /var/log/problem*.log input_type: log document_type: problem_log - paths: - /var/log/error-*.log input_type: log document_type: error_log multiline: pattern: '^[0-9]{6}_[0-9]{6}' negate: true match: after exclude_lines: ['^\n'] registry_file: /var/lib/filebeat/registry ############################# Output ########################################## output: logstash: protocol: https hosts: ["IP:5044"] worker: 1 compression_level: 3 loadbalance: true ssl: certificate_authorities: ["/etc/filebeat/logstash.crt"] ############################# Logging ######################################### logging: files: # The directory where the log files will written to. #path: /var/log/mybeat # The name of the files where the logs are written to. #name: mybeat # Configure log file size limit. If limit is reached, log file will be # automatically rotated rotateeverybytes: 10485760 # = 10MB # Number of rotated log files to keep. Oldest files will be deleted first. #keepfiles: 7 # Sets log level. The default log level is error. # Available log levels are: critical, error, warning, info, debug #level: error ``` What could be the cause? Maybe we should add/edit some parameters? Could you give us some tips, please? ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [December 28, 2016, 11:28pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/2 "2016-12-28T23:28:08Z")
The fail is due to EOF (end of file). That is either logstash itself or some other device/software did close the connection between beats and logstash. Maybe because logstash did take too much time. Which logstash, filebeat version are you using? If you're using LS 2.4 or 5.x, update the logstas-input-beats plugin to the most recent version. ---
**Author:** ![Gamer](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@Gamer](https://discuss.elastic.co/u/Gamer)\ **Post date:** [December 29, 2016, 3:44pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/3 "2016-12-29T15:44:09Z")
What's EOF? How does it correlate with the connection? Maybe we should set some timeout? The whole ELK stack consist of 5.x versions. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [December 29, 2016, 3:53pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/4 "2016-12-29T15:53:30Z")
[EOF](https://en.wikipedia.org/wiki/End-of-file) is short for `End Of File`. This means, no more content can be read/written as the underlying file descriptor has been close by the OS. Which versions are you using exactly? Have you update the logstash-input-beats plugin? Which exact version of logstash-input-beats have you installed? ---
**Author:** ![Gamer](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@Gamer](https://discuss.elastic.co/u/Gamer)\ **Post date:** [December 30, 2016, 2:06pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/5 "2016-12-30T14:06:29Z")
Hello! Logstash: 5.0.0 Filebeat: 5.1.1 Regarding EOF. It doesn't imply the actual end of a file (no more lines, no more data), but just the command from the system? So the file not necessarily can be read completely and some data may remain? Am I understanding that correctly? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [January 3, 2017, 8:39am UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/6 "2017-01-03T08:39:28Z")
The EOF you see is related to the connection to Logstash and the file you are reading itself. So you should investigate what goes wrong with the connection to LS. The version of the logstash-beats-plugin is also important here. Check if you see some errors / logs on the logstash side that could help here. Also there are quite a few existing topics with similar issues which were solved: [https://discuss.elastic.co/search?q=Failed%20to%20publish%20events%20caused%20by%3A%20EOF](https://discuss.elastic.co/search?q=Failed%20to%20publish%20events%20caused%20by%3A%20EOF) Perhaps one of these can help you solve your problem. ---
**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\ **Post date:** [January 3, 2017, 5:14pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/7 "2017-01-03T17:14:12Z")
Logstash 5.0.0 does not ship with most recent logstash-input-beats plugin. From [plugin changelog](https://github.com/logstash-plugins/logstash-input-beats/blob/master/CHANGELOG.md) it seems you want to have at least 3.1.11 installed. Use `./bin/logstash-plugin` to update the input plugin. ---
**Author:** ![Gamer](https://avatars.discourse-cdn.com/v4/letter/g/96bed5/32.png) [@Gamer](https://discuss.elastic.co/u/Gamer)\ **Post date:** [January 11, 2017, 3:17pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/8 "2017-01-11T15:17:04Z")
Oh, I didn't know about plugin updating. Ok, I'll give a try then. Thanks! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 8, 2017, 3:17pm UTC](https://discuss.elastic.co/t/filebeat-sends-logs-but-only-at-the-second-try/70082/9 "2017-02-08T15:17:46Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.