How to add a field/value from a previuos log

Elastic Stack Logstash
1

Hi all,

I am using ELK to store and search logs and I need some sort of aggregation over logs that are somehow correlated.
My use case is as follows:

Log #1 contains field A and field B
Log #2 contains field B and field C
Log #3 contains field B and field D
Log #4 contains field B and field E
...

I want to add field A and its value in all the next log lines containing field B. During my search I found the aggregate filter plugin which performs a similar action but only adds its result on the final event of a task. Instead I could use the elasticsearch filter plugin but I realized the ELK stack takes some minutes to index/make available an event, so the filter will not find Log #1 if invoked shortly after Log #1.

I know what I need is quite simple so I'm sure there is a straightforward solution, but which one?

Many thanks,
Daniele

2

I eventually used the aggregate filter with the following configuration, don't know if this is efficient though:

if [A] {
  # field A is present
  aggregate {
    task_id => "%{B}"
    code => "map['stored_field_A'] = event['A']"
    map_action => "create"
  }
}
if ![A] {
  # field A is NOT present
  aggregate {
    task_id => "%{B}"
    code => "event['A'] = map['stored_field_A']"
    map_action => "update"
    timeout => 86400
  }
}