@metadata and regexp named groups

Elastic Stack Logstash
1

in the url
https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html
we can see that we can use [@metadata][timestamp] in a grok:

filter {
  grok { match => [ "message", "%{HTTPDATE:[@metadata][timestamp]}" ] }
  date { match => [ "[@metadata][timestamp]", "dd/MMM/yyyy:HH:mm:ss Z" ] }
}

yet, nothing is said about regexp named groups and trying to use this

filter {
  grok { match => [ "message", "(?<[@metadata][timestamp]>[^ ]+)" ] }
  date { match => [ "[@metadata][timestamp]", "dd/MMM/yyyy:HH:mm:ss Z" ] }
}

fails... so how to use @metadata within a named group?

After knowing, i will probably open a bug to update the docs to include a example for named groups

thanks

2

I'm pretty sure you can't use nested fields (or even bracket/@ characters ) as capture groups in pure regex.
A possible alternative would be to capture it in a single temporary field and then use e.g. mutate to add it's value to '[@metadata][timestamp]'

3

That is almost what i'm doing right now... but then i have to remove the useless variable.

I was trying to save a few cycles (doing the drop of a temporary field) and space (waste disk space by storing both @timestamp and the event date field) by using the @metadata field to store the log date, as it is automatically discarded in the end.

In many millions events per day, a few optimizations like this always help... but if it is not possible, lets do the drop
thanks for the help

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.