Filebeat holds access to deleted/rotated files and impact disk space stats

[mt.michael]

Hi. We're using filebeat version 1.2.3 (amd64) on Ubuntu 12.04.5 LTS server.

We are using logrotate to rotate files (20 unique file names) hourly. So filebeat should monitor files with $filenames.log and $filenames.log.1

But when we've more data streaming to those files then it seems we've issue with disk space of getting full easily.

And when i run this command lsof +L1 /var I see alot of processs by filebeat access deleted files as that's what the command said (deleted).

Then those still count into the disk space stats. But everytime I restart filebeat those will be gone and free up disk space.

We tried these setting in yml files but doesn't seem help. Any suggestion as we don't want to keep restart filebeat service. Thanks.

  ignore_older: 5m
  close_older: 5m
  force_close_files: true

[ruflin]

There are some known issue in the 1.2 release related to open file handlers. Updating to the 5.0 release should solve your issues.

[sharon.c]

He is using logstash 2.3.4 and elasticsearch 2.4 in the stack, if he upgrade filebeat to 5.0, does he need to upgrade logstash and elasticsearch to 5.0 version also?

[ruflin]

No, filebeat is backward compatible. Please make sure to use in LS the most recent version pf the beast-input-plugin.