Im currently sending logs via a mounted volume.
docker-compose.yml looks like the following
version: '3'
services:
filebeat:
image: docker.elastic.co/beats/filebeat:6.3.0
container_name: filebeat
volumes:
- '/var/log/logstash/:/mnt/log:ro'
- './config/filebeat.yml:/usr/share/filebeat/filebeat.yml'
restart: always
network_mode: "host"
filebeat.yml looks like this
filebeat.prospectors:
- input_type: log
paths:
- /mnt/log/*.log
json.overwrite_keys: true
json.keys_under_root: true
fields_under_root: true
output.logstash:
hosts: ['localhost:5044']
Now the logs are generated by spring boot which will delete old logs after x amount of days. My concern is If I restart the machine or run a new container it will resend all the same logs.
Can I mount the filebeat "tracker file" or whatever it is called? So that it will persist reboots. Or is there another mechanism that will do the same thing?
Also did a search and this question has been asked before.. However couldnt really work out a solution based on the answers
https://discuss.elastic.co/t/filebeat-persist-processed-logs-if-docker-container-crashes/60089