Forwarding not updated CSV files

Elastic Stack Beats
1

Hi,

I have a software generating batches of logs in a CSV file once per hour. This CSV file is overwritten in one step instead of lines being appended to the previous file and is very formatted (all files are the exact same byte-level size). I tried a lot of configurations using ignore_older, clean_inactiveand close_eof as I found in this similar topic : https://discuss.elastic.co/t/monitor-a-set-of-json-files-in-filebeat/65198/2

None of my attempts were successful, filebeat refusing to harvest my files. Am I missing something ? How would you configure filebeat to handle such case ?

2

I assume the modtime gets updated when new content is added to the file. I would expect in case the file gets updated every hour, that something like scan_frequency: 10s, ignore_older: 1m, clean_inactive: 5m, close_eof: true would work.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.