Hi, We are using filebeat as a deamonset on kubernetes nodes to collect all application logs and sending them to logstash and then to elasticsearch. In our Prod environment, we have some pods writing huge volume of events like ~1000/sec. Filebeat is not able to send all those events from those pod…

Filebeat not able to send all logs

sandeepkanabar (Sandeep Kanabar) December 10, 2020, 10:04am 5

Have you tried increasing the value of close_timeout as described here.

Have you tried by disabling queue.mem and see if that helped?

Topic		Replies	Views
File beat stop harvesting logs from few files suddenly , works fine after restart Beats filebeat	2	421	May 23, 2019
Filebeat does not harvest all logs from application Beats docker , filebeat	1	657	April 18, 2022
Filebeat daemonset losing messages Beats docker , filebeat	2	788	June 22, 2022
Filebeat not collecting all logs Beats filebeat	1	291	March 28, 2023
Filebeat Pod not working properly Beats docker , filebeat , metricbeat	4	1086	March 10, 2021