Push logstash aggregate map as fields in final event

I am trying to use an aggregate filter to pull fields from several different but related logs into one "summary" log. There is a clear end_of_task event and I would like to "push"/add all of the fields in the map into the final end_of_task event.

Do I have to use a "event.set" command for each of the possible fields in the map?
event.set('field1', map['field1'])
I would prefer NOT to use this route as the number and names of the fields in the map can vary.
Can I iterate through all the fields in the map in some way?
Can I just push the whole map into the event as an array with event.set?
Something like this event.set('session_summary', map)?

Yes, you could do

    aggregate {
        task_id => "%{task}"
        map_action => "update"
        end_of_task => true
        code => 'map.each { |k, v| event.set(k, v) }'

You might also be able to do it using the push_previous_map_as_event option, but that would depend on your data.

Thanks @Badger!

Your comment about "push_previous_map_as event" produced another question, as this is my first time using the aggregate filter.
Due to the nature of my logs, it is very likely that I will have multiple maps being handled at the same time. My understanding of the "push_previous_map_as_event" is that it would push the previous map (and all associated data/fields) out of memory as soon as it saw a log with a different "task_id".
Is this correct?
And/or am I wrong in my understanding that this filter can handle multiple simultaneous maps?

That is my understanding of it too.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.