Aggregate fields from different event

Elastic Stack Logstash
1

Hi,

Please help me setting the filter plugin for the following scenario.
Say I have the following scenario

INFO - 12345 - TASK_START - start
INFO - 12345 - SQL - sqlQuery1 - 12
INFO - 12345 - SQL - sqlQuery2 - 34
INFO - 12345 - SQL - sqlQuery3 - 44
INFO - 12345 - TASK_END - end

If I am aggregating using taskid (12345), there are two things, I want to do.
One is just existence check whether sqlQuery1 happened or not
Other is basically collect all the fields, so that my final single document looks like.

(Assumption sqlQuery2 and sqlQuery3 always happen/present)

task id 12345
sqlQuery2Time 34
sqlQuery3Time 44
sqlQuery1PresentOrNot (boolean) true
sqlQuery1Time 12 (optional field )

Is Aggregate filter plugin the right one for my above scenario or any other plugins can help in as well.

Regards
Bharath

2

Aggregate is probably the one!

3

Hi,

Can aggregate filter help me in collecting different fields from intermediate lines/events and have it in the final aggregated event. I know it is mainly for simply adding up time based fields.

Thanks
Bharath

4

Hi,

I just went over the Use cases for this plugin

--extract error information in any task log line, and push it in final task event (to get a final document with all error information if any)

Can someone give me an example of how the "code" option should look like for the above scenario?

Thanks