Adding a field to a metrics event

Elastic Stack Logstash
1

Hi there, I am having a big issue with a really simple situation. I am trying to add a custom field to the event generated by the metrics filter plugin, however, it seems that the metrics plugin first creates the event and then adds the fields, therefore it doesn't have access to the same fields in previous events.
Here's a code example:

input{
    ......
    mutate {
        add_field => { "custom_field" => "some_custom_value" }   ### In reality this comes from my input directly (filebeat)
    }
}
filter{
    metrics {
                meter => ["%{count_on_this}"]
                add_tag => ["metrics"]
                add_field => { "my_field" => "%{custom_field}" }
            }
}

The resulting event is going to contain "my_field": "%{custom_field}" but will never resolve the value. Is there a way to make this work?

The limitation is that the value of my custom field comes from the received input, and therefore I cannot hardcode it into the metrics event.

Thanks in advance.

2

No, there is not. The metrics filter records creates a new event which contains metrics related to previously seen events. The add_field option can only reference fields on the newly created event. There is an issue for this, but the behaviour wll never change.

3

Gasp

How about using the aggregate filter or somehow defining the parameter as a constant?

4

An aggregate would suffer from the same issue. If you are metering events over a 15-minute period, which of those many events should be used to do the substitution of the field reference? logstash cannot assume that the referenced field will be the same on all the metered events. If it is then why use a field reference for it? Just use the string value in the add_field option.

1 Like
5

I see the point. Too bad then :frowning: If I find out a workaround I will make sure to update the post.

6

Just for curiosity purposes, I bypassed this limitation by adding an extra filter in the pipeline. Basically, I defined 3 conditionals based on the value of the field I wanted to set in metrics.

if "value1" in [custom_field] {
    metrics{
        .....
        add_field => { "custom_field => "value1" }
     }
} else if "value2" in [custom_field] {
    metrics{
        .....
        add_field => { "custom_field => "value2" }
     }
}

And then in a later step I process the event created by the metrics function. That did the trick for me.