# Filebeat still process old log file **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 7, 2019, 3:34am UTC](https://discuss.elastic.co/t/filebeat-still-process-old-log-file/163104 "2019-01-07T03:34:43Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![merceskoba](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@merceskoba](https://discuss.elastic.co/u/merceskoba)\ **Post date:** [January 7, 2019, 3:34am UTC](https://discuss.elastic.co/t/filebeat-still-process-old-log-file/163104/1 "2019-01-07T03:34:43Z")
Hello guys, I have a problem and just found the root cause yesterday. So, there were Filebeat and Logrotate in an instance. When Logrotate executed, Filebeat still read the old file. Even though the file is gone. This makes the disk size consumed hiddenly. How do i know? I did this, `lsof +L1` and then there was filebeat which was processing the old log file. After that, i need to `kill PID` or `service filebeat stop` and then `service filebeat start`. After this action, the disk size is normal back. Have you ever experienced this problem ? Do you have any idea to solve this without manual action ? Thank you for your time and help. ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [January 7, 2019, 6:16pm UTC](https://discuss.elastic.co/t/filebeat-still-process-old-log-file/163104/2 "2019-01-07T18:16:27Z")
Hello @merceskoba, To make sure all events are consumed, filebeat will by design keep file open until it reach `EOF` and the `close_inactive` value is triggered normally it's should be 5 minutes after Filebeat has completely read the file. All the of theses events are written to the log. Are you sure that Filebeat did consume all the events from the file? Can you share your harvester/input configuration? We can get a bit more log information if you start Filebeat using the "harvester" selector either by configuring it in the YAML or by starting filebeat with `-d "harvester"`. ---
**Author:** ![merceskoba](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@merceskoba](https://discuss.elastic.co/u/merceskoba)\ **Post date:** [January 9, 2019, 8:32am UTC](https://discuss.elastic.co/t/filebeat-still-process-old-log-file/163104/3 "2019-01-09T08:32:50Z")
[https://paste.ubuntu.com/p/2nw5m3TYGc/](https://paste.ubuntu.com/p/2nw5m3TYGc/) That's my config. And i just noticed this link [https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#close-inactive](https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#close-inactive) [https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#close-renamed](https://www.elastic.co/guide/en/beats/filebeat/6.2/configuration-filebeat-options.html#close-renamed) Should i add `close_inactive` or `close_renamed` ? If abc.log is rotated by logrotate and Filebeat is reading abc.log. Now, abc.log will be renamed to be abc.log.1 right ? is Filebeat still reading abc.log.1 ? I think 'yes', that's why `lsof +L1` detected Filebeat processed the previous log. But, it was stucked like Filebeat had never finished to read it. And my idea is add `close_renamed` if abc.log is renamed to abc.log.1, Filebeat will not read it but there is potential data loss caused the Filebeat doesn't read the log until EOF. ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [January 9, 2019, 7:33pm UTC](https://discuss.elastic.co/t/filebeat-still-process-old-log-file/163104/4 "2019-01-09T19:33:35Z")
`close_inactive` is already set to 5mins by default, so I would not change it and it should trigger closing the file when no events are read after 5mins. I would not introduce `close_renamed` because of the risk of losing events. > If abc.log is rotated by logrotate and Filebeat is reading abc.log. Now, abc.log will be renamed to be abc.log.1 right ? > is Filebeat still reading abc.log.1 ? > I think 'yes', that's why `lsof +L1` detected Filebeat processed the previous log. But, it was stucked like Filebeat had never finished to read it. Yes this is exactly what is happening and it's by design, so you don't lose events event. Usually when Filebeat keep FD open its because it didn't read the file completely and send the events to Logstash, this mean that you produce more events that your Logstash instance (or anything after) can ingest. Before using `close_renamed` and losing events I would encourage you to look at your logstash log to see if there is any errors and try to increase the ingestion capacity. ---
**Author:** ![merceskoba](https://avatars.discourse-cdn.com/v4/letter/m/57b2e6/32.png) [@merceskoba](https://discuss.elastic.co/u/merceskoba)\ **Post date:** [January 10, 2019, 9:53am UTC](https://discuss.elastic.co/t/filebeat-still-process-old-log-file/163104/5 "2019-01-10T09:53:45Z")
got it, buddy.... thank you for enlightening me 🙂 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 7, 2019, 9:53am UTC](https://discuss.elastic.co/t/filebeat-still-process-old-log-file/163104/6 "2019-02-07T09:53:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.