Config filebeat only to read last 5 minutes of log file

Elastic Stack Beats
1

Can filebeat read only last 5 minutes of log file?

From the doc only found tail_files but its for read latest line of log

2

ignore_older for files

It uses the file last modified time in the directory, not the timestamp on an event, not any timestamp of an event.

3

Thank you for you response

Actually I'm talking about event timestamp because my log only one file

4

This is some logstash filter ruby code to test timestamp skew. It is mostly to address devices that write logs at epoch_millis = 0 when they reboot until they get a good ntp time.

#
# if event timestamp skewed more than 3 days....   
#
ruby {
code => 'if ( Time.now.to_i - event.get("@timestamp").to_i ).abs > 259200 
             event.tag("timeskew")
             event.set("timeskew_original_timestamp", event.get("@timestamp"))
             event.set("@timestamp", LogStash::Timestamp.at(Time.now))
         end'
tag_on_exception => "rubyexception"

This could be modified to a shorter time value and do "event.cancel" if time is out of bounds.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.