Hey gang,
I'm running filebeat 1.3.1 and logstash 2.4.1 (haven't upgraded to newest because we are still using java7...) on an ubuntu instance on AWS. Once I deploy (we do a green-blue deployment scheme) with the AMI that contains filebeat and logstash, it works fine at least for a couple of hours (doing stuff on the instance shows in the logs). I'm shipping my logs to logz.io, filebeat and logstash are installed in the AMI, and communicate between them without TLS.
After a few hours (don't know how many exactly - the next day it happens for sure), logstash appears to be stuck, and I'm getting this in /var/log/logstash/logstash.stdout:
Failed (Connection refused - Connection refused). Sleeping for 0.02
Failed (Connection refused - Connection refused). Sleeping for 0.04
Failed (Connection refused - Connection refused). Sleeping for 0.08
Failed (Connection refused - Connection refused). Sleeping for 0.16
Failed (Connection refused - Connection refused). Sleeping for 0.32
Failed (Connection refused - Connection refused). Sleeping for 0.64
Failed (Connection refused - Connection refused). Sleeping for 1.28
Failed (Connection refused - Connection refused). Sleeping for 2.0
From that point on, that log contains lines just like the last one. I tried to play with keeping the file handlers open and so on, but nothing works. Can someone offer any piece of advice here?
Thanks!