File input not picking up copied or moved files


I'm successfully using logstash to parse json formated events and send them to elasticsearch. Each event is created in a seprate file. One event per file with .json extension.

Logstash is correctly picking up a file when i create it using "vi mydoc.json", paste in the content and save. However it does not pick up if I cp or mv a file.

The objective is to automatically copy files to a directory and then parse them by logstash.
Each file has a different name and size. I tried looking at logstash code to figure out what attribute it uses but couldnt find relevant code. I also tried deleting the .sincedb files but didn't help either.

The input config is as follows:

input {
file {
path => "/opt/rp/*.json"
type => "tp"
start_position => "beginning"
stat_interval => 1



How can I have logstash pick up copied files? What file stat attribute does it use to check if a file is new?


Logstash's file input (actually the Ruby filewatch library) tracks files via inode numbers. Copying or moving a file into a directory should make Logstash treat it as a new file unless there's inode reuse going on. You'll get additional debugging out if you start Logstash with --verbose.

Even I am facing the same issue where Log Stash is able to pickup the file which I created there but unable to pick up the files which is there by either copying or Moving or from Downloading.

Suggest some points.

Perhaps the files are old than 24 hours and you haven't adjusted the file input's ignore_older option. Otherwise the previously given advice of using the --verbose option is still relevant.