I was able to "reproduce" the Connection reset messages, but without it causing an issue. also used Rocky Linux 9.5. Firewall on/off did not appear to make any difference.
The slightly strange thing is the 2 sides disagree on keepalive timer status. I do see the TCP keepalive packets (and ACKs) with tcodump on both hosts. But logstash end seems to ignore them. And sometime after 3x 15 second countdowns I see at filebeat end, it (logstash) sends a RSET
The filebeat end shows this in netstat output:
$ sudo netstat -no | fgrep EST | fgrep 5044
tcp 0 0 192.168.178.67:45842 192.168.178.66:5044 ESTABLISHED keepalive (0.73/0/0)
But logstash end show this: (note the "off")
$ sudo netstat -no | fgrep EST | fgrep 5044
tcp6 0 0 192.168.178.66:5044 192.168.178.67:45842 ESTABLISHED off (0.00/0/0)
As long as the "pipe" is kept busy it's a non issue. I didn't see any lost logs, tested for just a little while of course.
If it's not busy, sure the RSET happens, but a new TCP connection is setup when required.
Below is the Wireshark view, the .66 is logstash and the .67 is filebeat (for the eagle eyed, my Wireshark source port / destination port columns are the wrong way round!)
The packet with length 754 is the actual log being sent from filebeat to logstash, the only one in that time window.
network experts might wish to weigh in.
