Issue with Aggregate task_id undefined variable

[ffernandez (Fernando)]

Hello!

I'm trying to use the aggregate filter plugin to merge a couple of events using the field "thread" obtained via the grok filter plugin. However, every time I try to run logstash, whether is from the cli or systemd start logstash I keep getting the following error:

Environment variable thread is not set and there is no default value given.

This is the relevant part of the filter code I'm using:

filter {
grok {
match => { "message" => "[%{TIMESTAMP_ISO8601:timestamp}] [thread %{POSINT:thread}] [client %{IPORHOST:clientip}:%{POSINT:clientport}]" }
}
aggregate {
aggregate_maps_path => "/var/log/logstash/aggregate_maps"
task_id => "${thread}"
code => "map['dynafed'] ||= event.get(['dynafed']['path'])"
push_previous_map_as_event => true
timeout => 1
}
}

I have tried changing it to "${[thread]} but then I get an error saying it needs to contain a dynamic expression like %{field}

I can use that "thread" variable within other plug-ins however, like mutate, or as a conditional (not that I want to do that just doing so to test it).

I have also tried something like below just to test:

aggregate {
add_field => "thread_id" = %{thread}
task_id => "%{thread_id}"
code => "map['dynafed'] ||= event.get(['dynafed']['path'])"
push_previous_map_as_event => true
timeout => 1
}

and I get the same "undefined variable" error as before. It's as if the aggregate block is working on a different scope.

So I'm not sure if I'm doing something wrong or maybe I found a bug? Let me know if you need any more info.

I'm using logstash 5.6.2 and the logstash-filter-aggregate is 2.6.3 on CentOS 7.3

Thanks in advance!

[ffernandez (Fernando)]

Never mind, I'm an idiot, should be %{thread} not ${thread}