Hi, My question is this, If a container running filebeat is lost and we launch a new container, the registry file of the old container will be lost too and the new container wouldn't know from where the harvester should read the new files which will cause inconsistent/ambiguous data in elasticsearc…

Hi Luvpreet You can mount the data directory to a persistent location to enable filebeat to resume reading the files. I have tested this on docker swarm, but you can extend this to k8 as well. Here is my swarm config: version: '3.7' services: beats: image: "myregistry/filebeat/7.2.0" vo…

NerdSec (Nachiket) July 22, 2019, 9:24am 5

Yes. That would be sufficient.

Also, in case you wish to delete the old files, have a look here:

1 Like

Topic		Replies	Views
Filebeat not considering registry and resending data Beats docker , filebeat	2	643	March 29, 2021
Filebeat container config Beats filebeat	7	1542	March 16, 2017
Registry not working in filebeat 7.16.1 Beats docker , filebeat	5	528	January 28, 2022
Filebeats is re-processing logs once it restarts Beats filebeat	6	4594	April 18, 2018
Filebeat: Persist processed logs if docker container crashes Beats filebeat	5	3746	September 29, 2016

Elasticsearch is a trademark of Elasticsearch BV, registered in the U.S. and in other countries
Trademarks
Terms
Privacy
Brand
Code of Conduct

Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.

Running Filebeat in docker with persistent registry file?

Related Topics