I am using logstash to parse test logs. Some test log files have multiple errors per file. Only the last error in each file contains the "scenario" information but I need all errors from the same file path to share the same "scenario" field data. I believe I can accomplish this using the mutate and aggregate filters but I am not sure how to implement this solution. My intuition is to use the mutate filter to add the scenario field if it doesn't already exist like this:
if ![scenario] {
mutate {
add_field => {"scenario" => ""}
}
}
and then use aggregate filter plugin to share the field data between all entries with the same file path like this:
aggregate {
task_id => "%{[log][file][path]}"
code => "map['sharedscenario'] ||= "";
map['sharedscenario'] += event.get('scenario');"
push_map_as_event_on_timeout => true
timeout_task_id_field => "[log][file][path]"
timeout => 300
timeout_tags => ['_aggregatetimeout']
}
So far this has not worked for me. If there is a better way to accomplish this or some obvious errors in my implementation please let me know. Thanks!