NEED TO READ ONLY THE NEWEST FILE IN EACH FOLDER

Elastic Stack Logstash
#1

Hi guys!!

I need to configure logstash to read only the newest file in folders, when I restart always read all files, and I couldn't understand why, files are in NAS, and I mount folders before start reading.

This is my last input config: (I tried with beginning.end, changing close_older and ignore_older values, tail or read mode)

input {
	file {
		mode => "tail"
		path => "/logs/**/*.ltf" 
		start_position => "end"
		sincedb_path => "/home/monava/db_logs"
    close_older => "1h"
    ignore_older => "1h"
    }
}

The newest file is always writing new lines.

Could some help with this??

#2

It is possible that if you re-mount the NAS it is being assigned a different device number. That would show up in the sincedb, which is just a text file.

#3

But when I restart logstash with the unit mounted, logstash read again all files...

#4

If you pick and example file name and search for it in the sincedb then what do you see?

You could try setting log.level to TRACE and seeing what the filewatch module reports (it is voluminous, so redirect logs to a file).

#5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.