# Way to identify matching grok pattern

**URL:** <https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082>\
**Category:** Logstash\
**Created:** [July 11, 2019, 5:36pm UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082 "2019-07-11T17:36:34Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![AmolB](https://avatars.discourse-cdn.com/v4/letter/a/aca169/32.png) [@AmolB](https://discuss.elastic.co/u/AmolB)\
**Post date:** [July 11, 2019, 5:36pm UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/1 "2019-07-11T17:36:34Z")

</div>

Hello Everyone,

I am using Grok filter plugin for Logstash (Logstash version 6.7.1) . My log line has several formats, so I have written more than 10 grok patterns for it. Now, I am interested in understanding the grok pattern which matched my log line. It could be 1 more field in elasticsearch index depicting matching grok pattern name or number so that I can create visualization on top of it.

The reason for it, if I know grok pattern which matches maximum of log lines, I can put that pattern at 1st position in the list of grok patterns. This could be performance gain for my use case, as most of my log line will match 1st pattern and will skip matching next grok pattern.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 11, 2019, 6:48pm UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/2 "2019-07-11T18:48:48Z")

</div>

In order to determine that you could split your grok so that each grok has a single pattern and adds a tag when it matches, then just count the tags in elasticsearch.

---

<div class="post-metadata">

**Author:** ![AmolB](https://avatars.discourse-cdn.com/v4/letter/a/aca169/32.png) [@AmolB](https://discuss.elastic.co/u/AmolB)\
**Post date:** [July 12, 2019, 5:59am UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/3 "2019-07-12T05:59:14Z")

</div>

Thanks Badger. But, if I have separate grok pattern then, my each logline will try to parse each of them which will be little overhead and performance hit for logstash.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 12, 2019, 4:53pm UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/4 "2019-07-12T16:53:27Z")

</div>

Correct, but you only need to do it once for a representative sample of data in order to determine the order of patterns in the consolidated grok.

---

<div class="post-metadata">

**Author:** ![AmolB](https://avatars.discourse-cdn.com/v4/letter/a/aca169/32.png) [@AmolB](https://discuss.elastic.co/u/AmolB)\
**Post date:** [July 15, 2019, 6:39am UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/5 "2019-07-15T06:39:30Z")

</div>

It seems, I can not collect such statistics on prod environment on continuous basis. I'll have to go with the approach you mentioned before I get into production.

Thanks.

---

<div class="post-metadata">

**Author:** ![BennyInc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bennyinc/32/21751_2.png) [@BennyInc](https://discuss.elastic.co/u/BennyInc)\
**Post date:** [July 15, 2019, 7:34am UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/6 "2019-07-15T07:34:23Z")

</div>

They currently match in order of appearance. You could split them up like @Badger suggested, adding tags for the individual grok filter. You could then also add a "grokked" tag, and run the next one conditionally:

```auto
if "grokked" not in [tags] {
  grok {
    match => [...]
    add_tag => ["this_grok_filter_id", "grokked"]
  }
}

```

---

<div class="post-metadata">

**Author:** ![AmolB](https://avatars.discourse-cdn.com/v4/letter/a/aca169/32.png) [@AmolB](https://discuss.elastic.co/u/AmolB)\
**Post date:** [July 15, 2019, 7:52am UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/7 "2019-07-15T07:52:26Z")

</div>

@BennyInc thanks. I am thinking on similar line 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 12, 2019, 7:52am UTC](https://discuss.elastic.co/t/way-to-identify-matching-grok-pattern/190082/8 "2019-08-12T07:52:34Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.