I checked these issues, but didn't find any solution. We already use input beats plugin 2.1.4. We have plain tcp between filebeat and logstash, no ssl, and no proxy. We tried to downgrade to logstash 2.1.9 as suggested in closed issue, but have found only 2.1.3, with no results after downgrade. I turned debug log level on filebeat, and here's what we noticed:
2016-03-11T15:22:43+01:00 DBG full line read
2016-03-11T15:22:43+01:00 DBG End of file reached: /usr/local/mpg/log/mpg.log; Backoff now.
2016-03-11T15:22:43+01:00 DBG connect
2016-03-11T15:22:43+01:00 INFO Connecting error publishing events (retrying): dial tcp x.x.x.x:5043: getsockopt: connection refused
2016-03-11T15:22:43+01:00 INFO send fail
2016-03-11T15:22:43+01:00 INFO backoff retry: 8s
2016-03-11T15:22:44+01:00 DBG Flushing spooler because of timeout. Events flushed: 654
2016-03-11T15:22:44+01:00 DBG full line read
There was no connection to logstash for only short period of time (reason unknown - EOF?), and after backoff, new lines are found. Logstash is running, nothing wrong.
2016-03-11T15:22:44+01:00 DBG End of file reached: /usr/local/mpg/log/mpg.log; Backoff now.
2016-03-11T15:22:45+01:00 DBG full line read
Yet another EOF, but new line is found immediately after. Logstash is running.
2016-03-11T15:22:50+01:00 DBG full line read
2016-03-11T15:22:51+01:00 DBG connect
2016-03-11T15:22:51+01:00 INFO Connecting error publishing events (retrying): dial tcp 10.50.110.20:5043: getsockopt: connection refused
2016-03-11T15:22:51+01:00 INFO send fail
2016-03-11T15:22:51+01:00 INFO backoff retry: 16s
2016-03-11T15:22:59+01:00 DBG Start next scan
2016-03-11T15:22:59+01:00 DBG scan path /usr/local/mpg/log/mpg.log
2016-03-11T15:22:59+01:00 DBG Check file for harvesting: /usr/local/mpg/log/mpg.log
2016-03-11T15:22:59+01:00 DBG Update existing file for harvesting: /usr/local/mpg/log/mpg.log
2016-03-11T15:22:59+01:00 DBG Not harvesting, file didn't change: /usr/local/mpg/log/mpg.log
2016-03-11T15:23:07+01:00 DBG connect
2016-03-11T15:23:07+01:00 INFO Connecting error publishing events (retrying): dial tcp 10.50.110.20:5043: getsockopt: connection refused
2016-03-11T15:23:07+01:00 INFO send fail
2016-03-11T15:23:07+01:00 INFO backoff retry: 32s
2016-03-11T15:23:09+01:00 DBG Start next scan
2016-03-11T15:23:09+01:00 DBG scan path /usr/local/mpg/log/mpg.log
2016-03-11T15:23:09+01:00 DBG Check file for harvesting: /usr/local/mpg/log/mpg.log
2016-03-11T15:23:09+01:00 DBG Update existing file for harvesting: /usr/local/mpg/log/mpg.log
2016-03-11T15:23:09+01:00 DBG Not harvesting, file didn't change: /usr/local/mpg/log/mpg.log
No connection to logstash once again, but this time no harvesting as well, file didn't change (EOF again?). After two backoffs, file still didn't change. At this point, a checked status of logstash returned 'not running', it was shutdown.