I am using filebeat-oss v7.12.0 in order to collect events from kubernetes pods using autodiscover.
It is expected to collect 5450 events from 5450 pods but fails to harvest for 150 pods and contains the following error in the logs,
FFAAAP-23-QA-3818 - 2021-08-11T20:48:49.592Z ERROR log/input.go:519 Harvester could not be started on new file: /var/log/containers/1628714791414-ne-ops-ne-backup-1628714791414-4lw6l_task-execution_tes-job-ctr-ffaaap23qa3818-1628714792064-a7080587d041af6298ba0b8da41dd423b964cedc252f5b7fa712de8dccecc528.log, Err: registry already stopped
I have gone through filebeat registry documentation but could not be able to find any parameters to avoid the registry from being stopped.
I came across "ignore_older", "clean_inactive" and "clean_removed" to clean files from registry, but not sure if they can minimize this issue.
Please help me in finding the cause for this issue along with a solution/workaround.