# Filebeat process holding deleted files

**URL:** https://discuss.elastic.co/t/filebeat-process-holding-deleted-files/43967
**Category:** Beats
**Tags:** filebeat
**Created:** [March 10, 2016, 1:07am UTC](https://discuss.elastic.co/t/filebeat-process-holding-deleted-files/43967 "2016-03-10T01:07:35Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![slickrick](https://avatars.discourse-cdn.com/v4/letter/s/48db29/32.png) [@slickrick](https://discuss.elastic.co/u/slickrick)
#### Post date: [March 10, 2016, 1:07am UTC](https://discuss.elastic.co/t/filebeat-process-holding-deleted-files/43967/1 "2016-03-10T01:07:35Z")

</div>

There was alert regarding disk space on our filebeat server

[root@filebeat3 ~]# df -h  
Filesystem Size Used Avail Use% Mounted on  
/dev/vda1 9.9G 3.5G 5.9G 38% /  
tmpfs 12G 0 12G 0% /dev/shm  
/dev/mapper/filebeat3 168G 137G 23G 87% /data

When I check /data it’s only using 45GB

-bash-4.1$ sudo du -sh  
46G

So I checked if there are files still open that are deleted.

[root@filebeat3]# lsof +L1 /data  
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME  
filebeat 29838 root 12r REG 253,0 5000018459 0 1703949 /data/scribe/publishing-service/publishing-service\_00003 (deleted)  
filebeat 29838 root 13r REG 253,0 5000004481 0 9306120 /data/scribe/merge-service/merge-service\_00000 (deleted)  
filebeat 29838 root 14r REG 253,0 5000005120 0 1703951 /data/scribe/idml-service/idml-service\_00020 (deleted)  
filebeat 29838 root 15r REG 253,0 5000014528 0 1703952 /data/scribe/publishing-service/publishing-service\_00004 (deleted)  
filebeat 29838 root 16r REG 253,0 5000016288 0 9306123 /data/scribe/basevariant-service/basevariant-service\_00001 (deleted)  
... many more lines similar...

[root@filebeat3]# ls -la data/scribe/attribute-extraction-kafka-connect-service/attribute-extraction-kafka-connect-service\_00000  
ls: cannot access data/scribe/attribute-extraction-kafka-connect-service/attribute-extraction-kafka-connect-service\_00000: No such file or directory

So filebeat is still holding the deleted files.. Restarted the daemon and the disk space are reclaimed. 😉

[root@filebeat3]# /etc/init.d/filebeat stop  
Stopping filebeat: [OK]

[root@filebeat3]# df -h  
Filesystem Size Used Avail Use% Mounted on  
/dev/vda1 9.9G 3.5G 5.9G 38% /  
tmpfs 12G 0 12G 0% /dev/shm  
/dev/mapper/filebeat3 168G 47G 113G 30% /data

Has anyone encountered this as well.. is there anyway to avoid having to restart filebeat process to clear the space?

Thanks,  
Rick

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [March 10, 2016, 11:49am UTC](https://discuss.elastic.co/t/filebeat-process-holding-deleted-files/43967/2 "2016-03-10T11:49:51Z")

</div>

Filebeat does keep files open until ignore\_older is reached. In version 1.2 the close\_older feature will be added: [https://github.com/elastic/beats/pull/718](https://github.com/elastic/beats/pull/718)

If you want to close files directly, use force\_close\_files but read the docs in detail to be aware of the potential side affects: [https://www.elastic.co/guide/en/beats/filebeat/1.1/configuration-filebeat-options.html#\_force\_close\_files](https://www.elastic.co/guide/en/beats/filebeat/1.1/configuration-filebeat-options.html#_force_close_files)

---

<div class="post-metadata">

### Author: ![tanmayghosh2507](https://avatars.discourse-cdn.com/v4/letter/t/8edcca/32.png) [@tanmayghosh2507](https://discuss.elastic.co/u/tanmayghosh2507)
#### Post date: [May 13, 2016, 6:31am UTC](https://discuss.elastic.co/t/filebeat-process-holding-deleted-files/43967/3 "2016-05-13T06:31:48Z")

</div>

I am facing the similar issue, I get disk usage alert quite frequently, and when I check and restart the filebeat, I get around 10-20 GB free space. So, what does actually happen when I restart the filebeat, why filebeat uses this much space, it should supposedly send the logs to logstash and not keep it to itself? Since, I cannot add "sudo service filebeat restart" into my script as a potential solution for the disk usage, how can I fix this issue without restarting the filebeat?

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [May 13, 2016, 1:40pm UTC](https://discuss.elastic.co/t/filebeat-process-holding-deleted-files/43967/4 "2016-05-13T13:40:21Z")

</div>

@tanmayghosh2507 Filebeat keeps the file handler open for a certain time to not have to open the file all the time and directly read new lines. The file is closed based on close\_older after a certain time. Which version of filebeat are you using? Could you share your config?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 9:52pm UTC](https://discuss.elastic.co/t/filebeat-process-holding-deleted-files/43967/5 "2017-07-05T21:52:01Z")

</div>


