We have discovered that filebeat loses first lines when file rotated.
For example we use the following config to get auditbeat logs and when file rotated (auditbeat restarted) we see that first lines missing (written in first 10 seconds = scan_frequency).
- type: filestream id: auditbeat-logs enabled: true paths: - /var/log/auditbeat/auditbeat-*.ndjson
Seems like filebeat uses tail_files strategy and I'm wondering how we can change this behavior and force it to read new files from the beginning.
We have checked all options and read all topics and posts like Log rotation results in lost or duplicate events
This issue make filebeat not reliable solution.