@ruflin I took a look through quite a few of those and they don't seem to be related to what I'm seeing. All of those seemed to have no success whatsoever sending lots from filebeat to LS. My logs are actually making it through but I'm also seeing these intermittent errors on a regular basis. Perhaps I need to set some tuning parameters or something like that?
Filebeat will keep resending the events until they get confirmed. It seems sometimes Logstash does not respond fast enough. That could be because of some network errors or that Logstash is too busy. Do you see it during peak load times? As long as the error is not constant it means from time to time there are issue but filebeat will make sure all events are delivered.
I would expect you to see some errors or at least info somewhere in the LS logs. Do you have something between FB and LS?
Logstash version? logstash-input-beats plugin version? Logstash closes inactive connections, leading filebeat to reconnect and continue sending logs. Some logstash-input-beats plugin versions seem to have a bug of closing an active connection, though. I think upgrading/downgrading logstash-input-beats plugin to version 3.1.10 should help (given you're using logstash 5.x).
The plugin closes inactive connections. The timeout for inactive connections can be adjusted in logstash. Maybe increasing the timeout to some much bigger value will help.
checkout bin/logstash-plugin tool for installing/uninstalling/updating plugins. As far as I can tell from other user reports, 3.1.10 seems to be most stable. Checking the changelog you might want to try 3.1.14 first, though. Sorry for the inconvenience, but I haven't run into this problem myself yet and can only recite from other users experiences so far.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.