We have experienced a lost data issue with filebeat > logstash.
Our Logstash server was down for 4 hours and some events have not been indexed.
The filebeat configuration is as follows :
scan_frequency: 30s ignore_older: 10m close_eof: true close_removed: true clean_removed: true clean_inactive: 15m
The indexed files are read only once because they are never updated once created.
I have identified some files that have not been indexed but I do not see them in filebeat logs (whereas I can see all other files).
What is the exact retry policy of filebeat ?
Can our configuration lead to data loss ?