We finally upgrade filebeat from 5 to 6.1.3, however, the same problem still persists.
(sorry I could not reply to the original thread since it is more than a month old.
We observed the same issue today even after we upgraded filebeat to 6.1.3.
(1) filebeat still holds on to its file descriptor when it is more than an hour after a file being rotated.
(since I have " close_inactive: 10m" in the config, filebeat should close the fd 10 minutes after a log file being rotated?
(2) After a file being deleted, filebeat still holds on to its file descriptor
I have close_removed: true, so this should not happen?
(Both observed from lsof)
Relevant filebeat config is:
closes the harvester/fd when a file is removed.
filebeat will focus on sending the current log files
btw, should I be more aggressive in these config setting, hoping even if one mechanism fails, another will pick up the slack?
e.g. , in addition to close_inactive: 10m
be more aggressive in the following?