Input file plugin - file is not read when modified

Elastic Stack Logstash
1

HI,

I am using file input plugin in logstash. My file gets replaced in regular time interval . I need the file to be read from the beginning each time it is replaced.
I have below setting . It reads from beginning only when the logstash is restarted. But after modification (added data in the beginning of file), it doesn't read it from the beginning.
file {
path => "/home/user1/test_2020-09-*.csv"
sincedb_path => "/dev/null"
start_position => "beginning"
}

2

Probably inode re-use. There are several github issues that discuss this. No simple solutions.

3

It reads the file from bottom on updating the file. Assuming that probably it is considering the file not yet closed and that's why reads from bottom as per tail mode, I also tried setting the close_older to a smaller interval (5 mins) and replaced the file after that.
But no luck !

4

Even after replacing the file after an hour, it didn't help.
Does filebeat has this capability of reading the file from the beginning every time when it is replaced ?

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.