Monitor a set of JSON files in Filebeat

[addy]

I want to use Filebeat to monitor my JSON files to read and forward it to Logstash/ElasticSearch.
In my use case, there are a set of JSON files created with a JSON object. After a period of time, the same set of files will be overwritten with new data which is supposed to be fetched by FIlebeat.
Ex: I have a file report1.json with following details: {"name":"test1", "result":"result1"}.
After some time, new details will be entered which will overwrite existing data. New data: {"name":"test2", "result":"result2"}.

In Filebeat 5, I tried with the new config to set close_eof: true & scan_frequency: 5m.
When I test it, the file handler is closed after reading the first set of data. After 5 minutes, file handler restarts and scans the file as its updated, but fails in offset comparison with the offset stored in registry file.
If instead of overwriting previous details,I add new data on a new line, it works perfectly fine.
I wanted to use the clean_* configs to clear my registry records, but since my data file isn't moving anywhere it will scan the existing data again. Also, data file will not be updated always in a fixed time interval.

Can anyone please help if there are any specific combination of configs that I can try.

Thanks
Adnan

[ruflin]

I think you are looking for ignore_older option which must be used in combination with clean_inactive. This obviously only works, if you really now the predefined period when files will be overwritten. close_eof could also be helpful in your case if the file is only written once.

The problem in the above case seems to be, that the new file is not necessarly smaller then the old file. Filebeat can detect truncated files to start reading from scratch, but as with log files assumed, new file is smaller then old file.