Write connection reset by peer - Filebeat > Logstash

[IcedGoblin]

Hi

After some help please with an error sending logs to Logstash using Filebeat. Configuration is working and logs are received however we are receiving the following error: -

2019-02-04T16:13:07.288Z        ERROR   logstash/async.go:235   Failed to publish events caused by: write tcp 192.168.167.144:56386->192.168.134.206:5044: write: connection reset by peer
2019-02-04T16:13:08.295Z        ERROR   pipeline/output.go:92   Failed to publish events: write tcp 192.168.167.144:56386->192.168.134.206:5044: write: connection reset by peer

Looking at other topics relating to the same error, I have tried adding the client_activity_timeout setting in Logstash to the Beats input. I initially set this to 60s but then increased to 120s and on the node I was testing on the errors stopped. However I noticed on another test ode that was not sending frequent events that the error is still logged on the first event or initial connection. The reason it solved the issue on the first node was due to this node sending events every 60s so I guess it was always using the same connection however on the node that was not sending regular events the connection would timeout and then a new connection is created hence the error then occurring.

I therefore have a few questions: -

1. Is this actually an error or just a warning that there was not an existing connection to reuse? If a red herring I could ignore these but then we see this failures in the Marvel monitoring which is unfortunate.

2. I can see people setting client_activity_timeout to either 0 or 86400, are there any downsides to this at all around resource utilization or port exhaustion?

3. Other than the client_activity_timeout setting is there any other way to resolve this issue?

We are currently using Filebeat and Logstash version 6.2.3.

Thanks in advance for any help anybody can offer and for taking the time to read and respond.