Log-stash does not trigger new copied files

Hello I have a box that receives files in a target folder .
I have the following config:
input {

 file {
    path => [ "/usr/local/ogr/target/*/*Group.json"]
    codec => json
    start_position => "beginning"
 }

}

output {
    elasticsearch {
        hosts => ["localhost:9200"]
        index => "groups"
    }

    stdout {}
}

Log-stash does not read the files by this config, However when i edit the file (adding a space), it triggers the file and process it.
How can I configure log-stash to read all the files as they move to my server (as they copied to that directory...)

Logstash probably think it already has processed the files (it probably did while you were trying things out). Deleting the sincedb file should help. See the file input documentation.

Thanks @magnusbaeck ,

I end up creating a bash file to add an empty line to the end of each file every 5 minutes, that makes the log-stash see a change in a file and triggers them.

*/5 * * * * sed -i -e '$a\' /usr/local/ogr/target/*/*/*Group.json

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.