Filebeat / kubernetes / data loss

Elastic Stack Beats
1

I am using following config https://raw.githubusercontent.com/elastic/beats/master/deploy/kubernetes/filebeat-kubernetes.yaml plus settings symlinks: true

I have container'd containers with rotated logs by default. 5 log files 3 of them are *.gz Files rotated by size exceeding 10 Mb. I have 30 harvesters and now i have 10000 d/s. Many containers write multiple logs per second

I observe data loss. Read offset is larger then file size.

I think I should use these parameters but I am confused which values should I set them to.

close_inactive: 1m
clean_inactive: 10m
ignore_older: 5m

2

Hey @elastic_user1,

Could you explain more about the rotation of your logs? Are they being rotated with copytruncate?