# Problem "grok"ing when there is a conditional part on it

**URL:** <https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919>\
**Category:** Logstash\
**Created:** [December 23, 2022, 2:24pm UTC](https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919 "2022-12-23T14:24:33Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![syunusic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syunusic/32/4131_2.png) [@syunusic](https://discuss.elastic.co/u/syunusic)\
**Post date:** [December 23, 2022, 2:24pm UTC](https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919/1 "2022-12-23T14:24:33Z")

</div>

If I have this text:

```auto
blah1=faa faa2 blah2=fee blah3=fii blah4=foo

```

how can I have the values of each variable (the one that are before the equal sign) with regex? I’ve tried with this grok:

```auto
blah1=(?<blah1>[^=]+) blah2=(?<blah2>[^=]+) blah3=(?<blah3>[^=]+) blah4=(?<blah4>[^\n]+)

```

but it doesn’t work fine when the text changes to something like this:

```auto
blah1=faa faa2 blah2=fee blah3=fii

```

I’ve tried with something like this:

```auto
blah1=(?<blah1>[^=]+) blah2=(?<blah2>[^=]+) blah3=(?<blah3>[^=]+)(| blah4=(?<blah4>[^\n]+))

```

And works.. BUT… that one doesn’t work wit the previous one (it only matches until the “blah3” part):

```auto
blah1=faa faa2 blah2=fee blah3=fii blah4=foo

```

So.. How should I create the grok in order to work in both cases?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [December 23, 2022, 3:00pm UTC](https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919/2 "2022-12-23T15:00:20Z")

</div>

If the message you want to parse has this format, you do not need to use grok, the message is key-value message and you can use the `kv` filter to parse it easily.

You will just need to use a `mutate` filter to change your message because you have unquoted values with spaces.

The following filter combination will parse your message:

```auto
ffilter {
    mutate {
        gsub => ["message", "(\S+=)", ",\1"]
    }
    mutate {
        gsub => ["message", " ,", ","]
    }
    kv {
        source => "message"
        field_split => ","
    } 
}

```

The first `mutate` with `gsub` changes your message from this:

```auto
blah1=faa faa2 blah2=fee blah3=fii blah4=foo

```

To this:

```auto
,blah1=faa faa2 ,blah2=fee ,blah3=fii ,blah4=foo

```

The second `mutate` with `gsub` will remove the extra space before the `,` so your ending message will be:

```auto
,blah1=faa faa2,blah2=fee,blah3=fii,blah4=foo

```

Now you can use the `kv` filter to parse your message setting the `field_split` to use the `,` and you will have your fields like this sample output:

```auto
{
         "blah3" => "fii",
         "blah4" => "foo",
      "@version" => "1",
       "message" => ",blah1=faa faa2,blah2=fee,blah3=fii,blah4=foo",
    "@timestamp" => 2022-12-23T14:56:10.353Z,
          "host" => "elk-lab",
         "blah2" => "fee",
         "blah1" => "faa faa2"
}

```

---

<div class="post-metadata">

**Author:** ![syunusic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syunusic/32/4131_2.png) [@syunusic](https://discuss.elastic.co/u/syunusic)\
**Post date:** [December 23, 2022, 3:06pm UTC](https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919/3 "2022-12-23T15:06:12Z")

</div>

Wow.. clever one the part of adding the commas. I'll try it. Thanks!  
But still.. I'm confised on why if I have a conditional (with "`(|...`" or even with "`(?:....)?`") it assumes always the conditional part (the blah4 in my example) doesn't exists.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [December 23, 2022, 7:16pm UTC](https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919/4 "2022-12-23T19:16:39Z")

</div>

> [@syunusic](#):
>
> How should I create the grok in order to work in both cases?

I totally agree with Leandro that grok is the wrong tool, but I found this an interesting question so I decided to answer it...

The problem with your second regexp

```
blah1=(?<blah1>[^=]+) blah2=(?<blah2>[^=]+) blah3=(?<blah3>[^=]+)(| blah4=(?<blah4>[^\n]+))

```

is that the blah3 pattern consumes everything that is not an equals sign (i.e. `fii blah4` and then the alternation in the blah4 pattern allows an empty match, so the overall pattern consumes

```
blah1=faa faa2 blah2=fee blah3=fii blah4=

```

from

```
blah1=faa faa2 blah2=fee blah3=fii blah4=foo

```

and leaves the `foo` unmatched. We can fix that by anchoring the end of the pattern using $ to force the whole line to be consumed.

```
    match => { "message" => "blah1=(?<blah1>[^=]+) blah2=(?<blah2>[^=]+) blah3=(?<blah3>[^=]+)(| blah4=(?<blah4>[^\n]+))$" }

```

results in

```
{
     "blah1" => "faa faa2",
     "blah4" => "foo",
     "blah3" => "fii",
     "blah2" => "fee"
}
{
     "blah1" => "faa faa2",
     "blah3" => "fii",
     "blah2" => "fee"
}

```

---

<div class="post-metadata">

**Author:** ![syunusic](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/syunusic/32/4131_2.png) [@syunusic](https://discuss.elastic.co/u/syunusic)\
**Post date:** [December 23, 2022, 11:53pm UTC](https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919/5 "2022-12-23T23:53:54Z")

</div>

Now I see @leandrojmp was totally right about not using grok in this case, but as you @Badger said, this was interesting to clarify.  
I just test @Badger 's solution and worked perfectly.  
Now, in production I'm using @leandrojmp 's approach and works like a charm.  
Thank you both. Everyday you learn something.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 20, 2023, 11:54pm UTC](https://discuss.elastic.co/t/problem-grok-ing-when-there-is-a-conditional-part-on-it/321919/6 "2023-01-20T23:54:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.