# Delete files using the "registry" file

**URL:** https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788
**Category:** Beats
**Tags:** filebeat
**Created:** [June 21, 2018, 5:41am UTC](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788 "2018-06-21T05:41:29Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)
#### Post date: [June 21, 2018, 5:41am UTC](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788/1 "2018-06-21T05:41:30Z")

</div>

Hi,

I have a use case in which, multiple large files are pushed on a server. These are read by Filebeat and sent to Logstash.

I have written a script to read the registry file and get the filenames that have been acknowledged by Fielbeat.

My question is, if it is safe to delete these files? When is a file actually updated in the registry? Are the files updated after they are completely processed by beats or are they updated when beats starts to process them?

Please suggest.

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [June 21, 2018, 8:16am UTC](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788/2 "2018-06-21T08:16:35Z")

</div>

Filebeat first saves a state of the file when it encounters it first. In this case the offset is 0, because there is not any messages which have been acknowledged. After the output returns the ACK, the number of ACKed messages in bytes are saved as the offset of the file. If the file is completely processed, meaning the last offset in the registry points to the end of the file, and no more lines are coming in, it should be safe to delete the file.

---

<div class="post-metadata">

### Author: ![NerdSec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nerdsec/32/22056_2.png) [@NerdSec](https://discuss.elastic.co/u/NerdSec)
#### Post date: [June 21, 2018, 10:55am UTC](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788/3 "2018-06-21T10:55:03Z")

</div>

Thank you for the clarification. Here is a python code for deleting files based on registry file information:

```auto
#!/bin/python
import json
import os

def watcher(filepath="/var/lib/filebeat/registry"):
    """Delete the files specified in the registry file."""
    f = open(filepath, 'r')
    a = f.readlines()
    f.close()
    b = json.loads(a[0])
    for x in b:
        try:
            if int(x["offset"]) == os.stat(x["source"]).st_size:
                os.remove(x["source"])
            else:
                print("File not processed, offset: ", x["source"])
        except Exception as e:
            print(e)

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 19, 2018, 10:55am UTC](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788/4 "2018-07-19T10:55:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
