Logstash isn't recognizing that a file has been updated

[TonyEnnis]

I have a largish file (4G). Each hour, more rows are added. I can process the large file when logstash is started, but regardless of what I do, logstash will not notice updates to the file.

The .sincedb_ file has a proper inode (?) value, and the 'last byte read' seems believable.

When I restart logstash, it will process the entire file. My config includes start_position => "beginning" but that shouldn't matter because of the sincedb file.

The exact same configuration works fine when I use a file that's tiny.

The updated date on the file, the inode value... they all make sense.

Finally, if I restart logstash, it will process the entire file again. My config includes start_position => "beginning" but that shouldn't matter because of the sincedb file.

So as I type this, it sounds like logstash isn't reading the sincedb file.

(Because nothing is easy, and because file doesn't accept URIs, the large file is in a shared folder mounted on my Redhat VM. That is, the file lives on some other server, if that matters.)

Here's my config:

input {
        file {
                path => ["/root/syslog.log"]
                type => "unknown"
                start_position => "beginning"
                ignore_older => 0
        }
}
output {
        rabbitmq {
                exchange => "logstash"
                exchange_type => "direct"
                key => "serviceMap"
                host => "rabbitmq.vm"
                }
}

EDIT - the ignore_older parameter above is new; it was in my config when I created the post. I saw another reference to this value and how it might be broken, so I added it. For the first time, logstash processed my big file when it was updated. At this time, I cannot tell if logstash is processing the entire file again or not.

EDIT 2 - the file is updated every hour. The 2nd time, logstash did not detect that the file had been changed.