Hi, My question is this, If a container running filebeat is lost and we launch a new container, the registry file of the old container will be lost too and the new container wouldn't know from where the harvester should read the new files which will cause inconsistent/ambiguous data in elasticsearc…

Hi Luvpreet You can mount the data directory to a persistent location to enable filebeat to resume reading the files. I have tested this on docker swarm, but you can extend this to k8 as well. Here is my swarm config: version: '3.7' services: beats: image: "myregistry/filebeat/7.2.0" vo…

NerdSec (Nachiket) July 22, 2019, 9:24am 5

Yes. That would be sufficient.

Also, in case you wish to delete the old files, have a look here:

1 Like

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Running Filebeat in docker with persistent registry file?