There are multiple files which I parse . however logstash seems to have missed 3 files , even though it has since_db entries for it it will not resume when i stopped and restarted logstash .
file {
path => "/home/test/Downloads/ELK/**/*.gz"
mode => read
type => "upi_logs"
file_completed_action => log
file_completed_log_path => "/home/test/Downloads/ELK/completedFileLogs.txt"
#sincedb_path => "/var/lib/logstash/plugins/inputs/file/.sincedb_53e23eb61e0f43f10fbe994941ff9669"
#start_position => beginning
#ignore_older => 0
}
As clear from above I am using read mode and write completed files in a txt file which does not have those 3 files . I tried enabling/disabling since_db path .