# Custom match pattern based on log message type

**URL:** <https://discuss.elastic.co/t/custom-match-pattern-based-on-log-message-type/269423>\
**Category:** Logstash\
**Created:** [April 7, 2021, 8:38am UTC](https://discuss.elastic.co/t/custom-match-pattern-based-on-log-message-type/269423 "2021-04-07T08:38:34Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![sergle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sergle/32/52241_2.png) [@sergle](https://discuss.elastic.co/u/sergle)\
**Post date:** [April 7, 2021, 8:38am UTC](https://discuss.elastic.co/t/custom-match-pattern-based-on-log-message-type/269423/1 "2021-04-07T08:38:34Z")

</div>

Hey there,

Would very much appreciate some input on something I am trying to achieve.

The setup:

`NLog -> (UDP) -> Logstash (ELK)`

The NLog is configured to send different message structures to Logstash, all in the form of:

`<date><msg_result><msg_checksum><msg_payload>`. The one thing that is constant on all messages is the message header structure which is `<date><msg_result><msg_checksum>`. The `<msg_payload>` varies what it holds.

The grok is configured as follows:

```
grok {
      break_on_match => true
      match => {
        "message" => "<%{TIMESTAMP_ISO8601:Date}><%{NUMBER:ProcessId}>%<{NUMBER:Result}><%{NUMBER:Checksum}>"
      }
      match => {
        "message" => "<%{TIMESTAMP_ISO8601:Date}><%{NUMBER:ProcessId}>%<{NUMBER:Result}><%{NUMBER:Checksum}><%{DATA:SomeData}>"
      }
      match => {
        "message" => "<%{TIMESTAMP_ISO8601:Date}><%{NUMBER:ProcessId}>%<{NUMBER:Result}><%{NUMBER:Checksum}><%{DATA:SomeData}><%{DATA:SomeMoreData}>"
      }
    }

```

Will grok only match the first part of the message (message header) and ignore the other since they all contain the same header `<date><msg_result><msg_checksum>`? The second question, how does one apply filter or match pattern based on `<msg_result>` before trying to match the payload?

I guess what I would like to know is how does grok digests and matches patters.

I have found somewhat similar situation to what I am trying to achieve here: [Use field value to match custom Grok pattern](https://discuss.elastic.co/t/use-field-value-to-match-custom-grok-pattern/157681)

Thanks a lot

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 7, 2021, 4:53pm UTC](https://discuss.elastic.co/t/custom-match-pattern-based-on-log-message-type/269423/2 "2021-04-07T16:53:46Z")

</div>

If you supply an option multiple times then logstash will combine them. It usually does that in the way you should expect, but sometimes not. As a result, I never use multiple instances of an option. I suggest you try

```
  match => {
      "message" => [
          "<%{TIMESTAMP_ISO8601:Date}><%{NUMBER:ProcessId}>%<{NUMBER:Result}><%{NUMBER:Checksum}><%{DATA:SomeData}><%{DATA:SomeMoreData}>",
          "<%{TIMESTAMP_ISO8601:Date}><%{NUMBER:ProcessId}>%<{NUMBER:Result}><%{NUMBER:Checksum}><%{DATA:SomeData}>",
          "<%{TIMESTAMP_ISO8601:Date}><%{NUMBER:ProcessId}>%<{NUMBER:Result}><%{NUMBER:Checksum}>"
      ]
  }

```

That will try to match the first pattern first, then move on to the second if the first fails to match, then try the third.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 5, 2021, 4:54pm UTC](https://discuss.elastic.co/t/custom-match-pattern-based-on-log-message-type/269423/3 "2021-05-05T16:54:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.