Hello, Filebeat experts, we just encountered a disk usage issue after deploying filebeat to production a few weeks. We set up filebeat to ship log files (file with extension .log) to kafka. The log files are rotated every day, and appended with the timestamp (e.g., XXX.log.datestamp). Log files m…

[image] Filebeat is not closig the file descriptor once the file is deleted/renamed Beats Have the same issue. Old logs are rename each hour, then delete after hour of inactivity by script. But filebeat still keep all or many of them open. Can't reproduce in test lab, only…

Have you had a look at the different close settings ? Note, filebeat can only close files if it's not being blocked by unresponsive outputs.

@steffens , could you elaborate on "filebeat can only close files if it's not being blocked by unresponsive outputs"? Can you provide an example of "unresponsive outputs"? Re. the close settings, by default, close_inactive is 5m, which is true in our case per the logging (Closing because close_ina…

At this location filebeat publishes a new event. If he spooler has become full, waiting for events to be ACKed, the sending will block on a go-channel. Only after having the current event published will the harvester check if the file needs to be closed. If the harvester is waiting for new lines (r…

Discuss the Elastic Stack

Filebeat might erroneously hold on to the file descriptor?

Elastic Stack Beats

vkakhnych (Vkakhnych) October 24, 2017, 12:44pm 2

Topic		Replies	Views
Filebeat holds on to a file descriptor too long even after a file was deleted? Beats filebeat	9	3091	March 5, 2018
Filebeat is not closig the file descriptor once the file is deleted/renamed Beats filebeat	27	7402	December 25, 2017
Filebeat is not closig the file descriptor and the file descriptors are dangling, when the file is deleted/renamed Beats filebeat	4	727	April 12, 2018
Filebeat is not closig the file descriptor, when the file is deleted/renamed Beats filebeat	3	700	February 16, 2018
Filebeat holds on to a file descriptor too long Beats filebeat	2	561	April 20, 2018

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Filebeat might erroneously hold on to the file descriptor?

Related topics