Help with Aggregate filter

Elastic Stack Logstash
1

aggregate
{
timeout_tags => ['_aggregatetimeout']
task_id => "%{[log][file][path]}"
code => 'map["build_name"] = event.get("[build_name_long]") ; map["build_link"] = event.get("[build_name_start]") ; event.set("build_name", map["build_name"])'
map_action => "create"
push_map_as_event_on_timeout => true
add_tag => ["start_aggregate"]
}

aggregate
{
task_id => "%{[log][file][path]}"
code => 'event.set("build_name", map["build_name"]) ; event.set("build_link", map["build_link"])'
add_tag => ["do_aggregate"]
}

Logstash version - 6.8.17
I am trying to fetch build_name_long and build_name_start value from grok filter and those are coming correctly, but on second filter while trying to set those value to build_name and build_link fields name not getting correct results. Can anyone please help me on this?

2

If you run this configuration, what result do you get?

input { generator { count => 1 lines => [ '{"id": 1, "a": 2}', '{"id": 1}' ] codec => json } }
filter {
    aggregate {
        task_id => "%{[id]}"
        code => 'map["a"] = event.get("[a]")'
        map_action => "create"
        push_map_as_event_on_timeout => true
        timeout => 10
        add_tag => ["start_aggregate"]
    }
    aggregate {
        task_id => "%{[id]}"
        code => 'event.set("a", map["a"])'
        add_tag => ["do_aggregate"]
    }
}
output { stdout { codec => rubydebug { metadata => false } } }
3

Hello,
Thanks for your help. I tried your suggested code and I am getting below result :
{
"sequence" => 0,
"tags" => [
[0] "start_aggregate",
[1] "do_aggregate"
],
"@version" => "1",
"id" => 1,
"a" => 2,
"host" => "VI281154",
"@timestamp" => 2022-02-25T07:55:41.632Z
}
{
"sequence" => 0,
"tags" => [
[0] "do_aggregate"
],
"@version" => "1",
"id" => 1,
"a" => 2,
"host" => "VI281154",
"@timestamp" => 2022-02-25T07:55:41.647Z

4

That is what I would expect. In that case I cannot explain why your aggregates do not work the same way.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.