Logstash missing files

[jamesp220291 (james)]

Hi

See my logstash input config -

#input {
file {
path => "/data/logs/*"
mode => "read"
file_completed_action => "log"
file_completed_log_path => "/data/logstash-db/17_file_completed_log"
}
}

I placed 4389 files in /data/logs/

When doing cat /data/logstash-db/17_file_completed.log | wc -l

it is saying 3211, so it is missing/ignoring 1178 of the files.

How do i determine why it is decided to completely ignore these files?

I also get different results each time, if i delete the index/clear the sincedb ect and re-copy them, sometimes it will do all of them, sometimes less.

[Badger]

All I can suggest is to enable '--log.level trace' and wade through tens of thousands of lines of filewatch trace messages to see what is different about the files that it does not read.

[jamesp220291 (james)]

Hey @Badger

How do i go about doing this?