Do Grok patterns have a place in the if =~ conditional matching?

Elastic Stack Logstash
1

I have the case where..

Sometimes my log has a block of JSON.
Sometimes that block of JSON has a field (Lets call it "IPAddress")
Sometimes the field "IPAddress" has a valid IP, other times, I've seen it with the value "unknown" or an empty string.

I want to..

If the "IPAddress" field exists..
If the field contains a valid IP address..

THEN, apply the geoip filter to it.

I know that there is a %{IP} defined pattern for Grok.
What I would like to do is something similar to..

if [IPAddress] =~ %{IP} {
  #Do stuff
}

However there are few caveats I've seen.

  1. It doesn't seem to work that way. In place of that, I found the base regex that comprises the %{IP} pattern, and now have done this:
    if [IPAddress] =~ /Long horrible string of regex.. seriously like 8 lines/
    That works. But, its messy.

  2. What kind of performance impact does this present, if any? And would any performance be gained by replacing the long regex string with %{IP}?

So overall, I would argue that a more readable config file, using Grok patterns for situations like this, is a benefit.
What do you guys think?

And, maybe this is just a horrible way to approach my problems. What might be some other ways to solve this?
Thanks!

Edit: Post had multiple items as a numbered list... all with the same number. Fixed. Also changed some words and emphasized some text for clarity

2

It doesn't seem to work that way.

Yeah, I'm pretty sure the regexps in conditionals don't support grok patterns.

And would any performance be gained by replacing the long regex string with %{IP}?

I'd assume the IP pattern in the end expands to more or less the same expression that you're currently using, so no.

But why have such a strict regexp? Is it really likely that the IPAddress field frequently will contain strings that match a simple expression like ^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$ yet aren't valid IPv4 addresses?

1 Like
3

So.. that's definitely a valid question.
Originally I was trying to match with a grok filter, using %{IP}. I carried that over when I ended up making it an IF statement.

My thinking is.. if I am going to end up matching a field with %{IP} at any point, I would want my if statement to be an exact replica of that match. Not leaving any gaps. I didn't have any specific examples in mind, it was just a way to not leave any ground open for unexpected errors in the future.