Logstash File Input Won't Re-read Files

[mfisher (Max)]

I've configured logstash to read all .system files from a specific directory and it works for new files.

However after the file is read and indexed it won't re-read a file with the same file name even if I edit it or delete and replace it.

I thought it might have to do with the sincedb however I specify the path to /dev/null and it still has the same behavior.

My input section for logstash.

input {
       file {
                mode => "read"
                path => ["/etc/logstash/config/sysinfo/*.system"]
                start_position => "beginning"
                sincedb_path => ["/dev/null"]
                type => "system-info"
       }
}

[elasticforme (Sachin)]

it has to do with inode.

https://discuss.elastic.co/t/file-updates-in-read-mode/202730

[Badger]

@mfisher

I thought it might have to do with the sincedb however I specify the path to /dev/null and it still has the same behavior.

The sincedb_path controls where logstash persists the sincedb when it is stopped. The sincedb always exists for the time the file input running. I agree this is probably inode re-use. Given that in read mode the default operation is delete, it is not possible for the re-used inode to contain the same file. There is an issue open to get the sincedb cleaned when the file is deleted.

[mfisher (Max)]

Gotcha, thanks again.