I have a problem inputing file on logstash

I'm trying to input the file which name is same for everytime
It reads well but I'd like to output only the last line appended in that time

my pipeline configuration is here

input {
file{
path => "/home/lisa/elasticstack/data_store/hmi/hmi.csv"
start_position=>"beginning"
type => "hello"
}
}

filter {
if [type] == "hello" {
csv{
separator=>","
columns=>["message number","timestamp","state","message text"]
}
grok { match => [ "path", "/(?[^/]+).csv" ] }
}
}

output {
stdout { codec => rubydebug }
}

the problem is it reads and ouput whole content of overwritten file..
I just want to use only appended last line

Is there any way to use overwritten file(only appended but overwritten) in this case?..

Best Regards

How are you appending a line to the end of the file? If you use an editor this typically results in a new file being stored with the same name as the old, which means Logstash will reprocess it as the inode changed. Instead try something like this: echo "1,2019-10-08:05:00:00,CA,test message" >> /home/lisa/elasticstack/data_store/hmi/hmi.csv

1 Like

I think this is the best way!

But there was rhe reason why I had to do it...

Thank you for answer

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.