Can you reference a dynamic variable in the grok filter?

I have been trying to split up the input of a script from execbeat.
So far everything has been going smooth.

I have input of similar fashion.

rpm-2109130090009900.x86-64 Thu 2019 ... and so on.

So basically an information of the installed RPM package and the DATE it has been installed.
Now I was trying the following grok in logstash:

if [type] == "execbeat" {
                split {
                        field => "[exec][stdout]"
                }
                grok {
                        match => {
                                "[exec][stdout]" => "%{NOTSPACE:rpm} %{GREEDYDATA:installdate}"
                                "rpm" => "%{GREEDYDATA}-%{INT:rpmtimestamp}"
                        }
                }
        }

I was thinking that I could access the 'rpm' variable for the 2nd match but it doesn't appear to work. Is there a better way to make this work or do I have an error somewhere? If i use the second grok on [exec][stdout] then I get the rpmtimestamp variable in the index, so the grok pattern itself is correct.

The solution is the following:

        if [type] == "execbeat" {
                split {
                        field => "[exec][stdout]"
                }
                grok {
                        match => {
                                "[exec][stdout]" => "%{NOTSPACE:rpm} %{GREEDYDATA:installdate}"
                        }
                        match => {
                                "[exec][stdout]" => "%{GREEDYDATA}-%{INT:rpmtimestamp}"
                        }
                        break_on_match => false
                }
        }

So you can't define multiple references to "[exec][stdout]" in the same MATCH block, thus you split the match blocks into multiple once. But here you have to be careful as GROK has the default behaviour to break/stop after the first successful match. As such it is necessary to add the last line:
break_on_match => false

If you want to match a field against multiple patterns the standard way to do it would be

match => {
    "[exec][stdout]" => [ 
        "%{NOTSPACE:rpm} %{GREEDYDATA:installdate}",
        "%{GREEDYDATA}-%{INT:rpmtimestamp}"
    ]
}

Combining multiple occurrences of a filter option (two match options) works differently in different versions of logstash and will sometimes do very unexpected things. I advise against doing it.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.