Reset file offset once the harvester reaches end of file

Elastic Stack Beats
1

Is there a configuration property available in filebeat to start the harvester always from the beginning? The harvester should start from the beginning of the file irrespective of the update.

We have specific logs (non-timestamp based) that get generated less frequently (say every day at 00.00 AM). The changes can be at any level of the file like removal of a few records in the middle etc.,

We need to configure filebeat in such a way that it always starts from the beginning of the file regardless of the change (could be any crud operation). I know there will be duplication of data with this scenario but that is being handled at a later stage (logstash)

Thanks!

2

I suggest experimenting with close_* options. This isn't a common use case as we used to recommend users data deduplication :slight_smile:

Please take a look at the doc page: Log input | Filebeat Reference [7.15] | Elastic

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.