Hi All. I am using file input plugin to read logs. My config looks like below:
input {
file {
path => "/opt/gtal/iptal/elasticsearch/app/logstash/stage/CVMQA_UAT_STATS-*.txt"
codec => multiline { pattern => "^Spalanzani" negate => true what => previous auto_flush_interval => 1 multiline_tag => "" }
start_position => "beginning"
sincedb_path => "/dev/null"
}
I noticed that new file does get read every time but for some reason it reports "no change". Trace was enabled and following is seen:
[2021-06-15T15:09:47,095][TRACE][filewatch.tailmode.processor][main] Active - no change {"watched_file"=>"<FileWatch::WatchedFile: @filename='CVMQA_UAT_STATS-15062021.txt', @state='active', @recent_states='[:watched, :watched, :ignored, :watched]', @bytes_read='7469', @bytes_unread='0', current_size='7469', last_stat_size='7469', file_open?='true', @initial=false, @sincedb_key='1613141125 0 64770'>"}
This report from filewatch seems untrue as logstash seems to be reading file with this filesize for the first time.
Is there a way to "force" logstash to read/ parse every file regardless of filesize or inode value?