I am using filebeat to forward logs from a pair of Varnish servers, and these produce a fairly large amount of data pretty quickly.
Because we don't want to keep this stuff, I've set up a one-hour log-rotation schedule for the file, so I end up with a bunch of files like:
-rw-r--r-- 1 root root 25M Dec 3 09:56 /var/log/varnish/varnishncsa.log -rw-r--r-- 1 root root 26M Dec 3 06:01 varnishncsa.log.20151203.1449122461 -rw-r--r-- 1 root root 29M Dec 3 07:01 varnishncsa.log.20151203.1449126061 -rw-r--r-- 1 root root 41M Dec 3 08:01 varnishncsa.log.20151203.1449129661 -rw-r--r-- 1 root root 66M Dec 3 09:01 varnishncsa.log.20151203.1449133261
My filebeat.yml is set at the top to ignore_older: 1h
filebeat: ignore_older: 1h prospectors: - type: log document_type: varnish paths: - "/var/log/varnish/*.log"
But I still frequently find that the disk has filled up because filebeat is holding files open. As soon as I restart the filebeat service, all that space comes back.
I'll try and update this post with more details the next time this happens, such as the open-file list for filebeat.
filebeat version 1.0.0 (amd64)