What I assume is like below,
The filebeat process launched in docker container, with NATed network with 5 of output logstash hosts.
At the begining it has 5 of established connection with logstash.
Single line of log sent to logstash, then window size become 15 from the initial window size 10, And MaxOkWindowSize also become 10.
Several hours later(conntracks tcp established timeout, or logstash idle timeout) without incoming log, connection to logstash is disconnected but socket is still opened.
After getting new single line of log. filebeat fails to send 5 times, windowSize shrinks to 1.
Reopen connection, succeed to send log.
Now windowSize is 1 and maxOkWindowSize is 2.
Again several hours pass, it fails again.
Now windowSize and maxOkWindowSize become 1.
It never be changed without TTL (seems it is available from v 6.0)
We use filebeat version 5.4.2 with below output configuration.
I think to solve it ....
- Enable TTL to refresh window size without code write.
- Implement TCP Keep-Alive
- Implement TODO part
- Try other heuristic workaround ... like
- add 1 to maxOkWindowSize if it has same value to windowSize