I have two log files. I am correlating these two files by timestamp using aggregate filter.
Where there is timestamp match between two files, correlation is working fine.
But there are scenarios where either of the two files may not have error for the same timestamp. In that case, I need to populate certain field values with ‘NA’ as value.
I have tried certain solutions but it impacts the aggregation scenarios also and correlation fails.
Can you suggest how can I handle non-aggregation scenarios in the same logstash.conf file if the event does not have any correlation( event with same timestamp in other log file)?
Use a timeout and in the timeout_code set the fields you want to "NA". Example 3 in the documentation shows how to use push_map_as_event_on_timeout.
Thanks for the solution Badger. It worked.
But sometimes I could see that even in case of correlation, I am getting “NA”. May be aggregate filter is reading files in separate threads. Although I have set worker as 1.
Moreover, if the log file is having only a small change, suppose, single line error is induced, aggregate filter fails to correlate. For testing, I am manually inducing errors in both of my log files.
Can you please suggest some solution?
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.