# Running Filebeat in docker with persistent registry file?
**URL:** https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595
**Category:** Beats
**Tags:** filebeat
**Created:** [July 22, 2019, 7:40am UTC](https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595 "2019-07-22T07:40:53Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)
#### Post date: [July 22, 2019, 7:40am UTC](https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595/1 "2019-07-22T07:40:53Z")
Hi, My question is this,
If a container running filebeat is lost and we launch a new container, the registry file of the old container will be lost too and the new container wouldn't know from where the harvester should read the new files which will cause inconsistent/ambiguous data in elasticsearch. It will read the whole file again. So, how should I go with creating a consistent registry file?
Should I also mount `/var/lib/filebeat/` to some persistent volume?
Right now, I am thinking in kubernetes(AWS EKS) context. I have a persistent volume(AWS EFS). I have created a `pvc`(persistent volume claim) for each application, lets say `app1-pvc` for app1. This `pvc` is mounted to `/var/log/app1/` location both on application and filebeat container. Filebeat is reading input from the file `/var/log/app1/api_info.log`. So, if my filebeat container gets restarted, the registry file will be lost and the whole api\_info will be read again.
Am I thinking in the right way? If yes, then what are the ways to avoid the scenario which I'm facing here? If no, how does it prevent the above scenario? Or where I'm thinking wrong?
---
### Author:  [@NerdSec](https://discuss.elastic.co/u/NerdSec)
#### Post date: [July 22, 2019, 8:20am UTC](https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595/2 "2019-07-22T08:20:35Z")
Hi Luvpreet
You can mount the data directory to a persistent location to enable filebeat to resume reading the files. I have tested this on docker swarm, but you can extend this to k8 as well.
Here is my swarm config:
```auto
version: '3.7'
services:
beats:
image: "myregistry/filebeat/7.2.0"
volumes:
- /filebeat/conf/filebeat.yml:/etc/filebeat/filebeat.yml:ro
- /filebeat/volumes/data:/usr/share/filebeat/bin/data:rw
command: "/usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml"
```
The volume and command could change depending on your image or os. Hope this helps.
Cheers!
---
### Author:  [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)
#### Post date: [July 22, 2019, 8:58am UTC](https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595/3 "2019-07-22T08:58:52Z")
Thanks for the response @NerdSec
My data directory is `/usr/share/filebeat/data`,
```
sh-4.2$ pwd
/usr/share/filebeat/data
sh-4.2$ ls
meta.json registry
```
So, I guess it will be fine if I only make `/usr/share/filebeat/data` directory a persistent directory?
---
### Author:  [@sLuvpreet33](https://discuss.elastic.co/u/sLuvpreet33)
#### Post date: [July 22, 2019, 9:21am UTC](https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595/4 "2019-07-22T09:21:35Z")
Cool, I tried it and it worked. I only mounted the data directory.
---
### Author:  [@NerdSec](https://discuss.elastic.co/u/NerdSec)
#### Post date: [July 22, 2019, 9:24am UTC](https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595/5 "2019-07-22T09:24:08Z")
Yes. That would be sufficient. 🙂
Also, in case you wish to delete the old files, have a look here:
> [@Delete files using the "registry" file](https://discuss.elastic.co/t/delete-files-using-the-registry-file/136788):
>
> Hi, I have a use case in which, multiple large files are pushed on a server. These are read by Filebeat and sent to Logstash. I have written a script to read the registry file and get the filenames that have been acknowledged by Fielbeat. My question is, if it is safe to delete these files? When is a file actually updated in the registry? Are the files updated after they are completely processed by beats or are they updated when beats starts to process them? Please suggest.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 19, 2019, 9:24am UTC](https://discuss.elastic.co/t/running-filebeat-in-docker-with-persistent-registry-file/191595/6 "2019-08-19T09:24:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.