I am using ELK to store and search logs and I need some sort of aggregation over logs that are somehow correlated.
My use case is as follows:
Log #1 contains field A and field B
Log #2 contains field B and field C
Log #3 contains field B and field D
Log #4 contains field B and field E
I want to add field A and its value in all the next log lines containing field B. During my search I found the aggregate filter plugin which performs a similar action but only adds its result on the final event of a task. Instead I could use the elasticsearch filter plugin but I realized the ELK stack takes some minutes to index/make available an event, so the filter will not find Log #1 if invoked shortly after Log #1.
I know what I need is quite simple so I'm sure there is a straightforward solution, but which one?