My boss argues that the TCP protocol in filebeat increases too many connections that will fill the connection pool and block other processes, so he advised us to use UDP. My question is whether he is correct and whether filebeat can use the UDP protocol. How can I persuade my boss that using TCP is ok?
Are these logs not important to anyone?
UDP is light-weight and fast, but does not offer any kind of delivery guarantee or retry mechanism. If you are fine losing some, or potentials all, of the data it is a very efficient protocol. This lack of delivery guarantee is also why it is not supported by Filebeat, as data loss is generally not acceptable when it comes to logs.
Emmm, then will the TCP protocol cause some problem such as blocking other processes? Does the filebeat's TCP way have some methods to limit the number of connecctions? As some more important systems are runing in the same server, we are concerned that the potential faults of the filebeat's TCP way may make the important systems unavailable.
Is this concern based on tests you have performed or speculation?
1 Like
There has been a recent bug fix to the the logstash plugin to fix connections being left in CLOSE_WAIT state on the logstash side: https://github.com/logstash-plugins/logstash-input-beats/blob/master/CHANGELOG.md#504
TCP protocol cause some problem such as blocking other processes
Why you think filebeat using TCP will block other processes?
TCP way have some methods to limit the number of connections
filebeat uses one TCP socket per host/worker. On I/O errors, filebeat closes the socket and creates a new one.
This topic was automatically closed after 21 days. New replies are no longer allowed.