# Best way to parse multiple message patterns

**URL:** <https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836>\
**Category:** Logstash\
**Created:** [May 16, 2022, 2:54pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836 "2022-05-16T14:54:36Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Thuunder7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thuunder7/32/97482_2.png) [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\
**Post date:** [May 16, 2022, 2:54pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/1 "2022-05-16T14:54:36Z")

</div>

Hello,

I want to ask if it is possible to have multiple dissect patterns?  
I know i can create conditionals based on the "\_dissectfailure" and create another dissect to parse other patterns, but this doesn't prevent the previous dissect to print a warning message.  
I am currently facing an issue while using `dissect { mapping => ... }`, when the field message contains a pattern different than the one i defined it always prints a warning message on the logstash logs. Is there a way to silence this logs?

Regards,

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [May 16, 2022, 3:53pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/2 "2022-05-16T15:53:21Z")

</div>

I think is it's not possible because there is no _match_ plugin as in _grok_. You can:  
a) Use IF

```auto
add if fielda=="value1" {
 dissect {
    mapping => {"field1 filed 2...."}
}
else if fieldb=="value1" {
 dissect {
    mapping => {"field1 filed 2...."}
}

```

Field A anf B can get by grok or similar .  
b) use field count, then if... else base on field numbers.

---

<div class="post-metadata">

**Author:** ![Thuunder7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thuunder7/32/97482_2.png) [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\
**Post date:** [May 18, 2022, 11:05am UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/3 "2022-05-18T11:05:20Z")

</div>

Yes but sadly i don't have a field to make a flow like you did. That would be the best approach.  
Do you know if GROK is more silent than Dissect? Since i don't have a field to make IFs, i would like to at least silence these logs.  
I have heard that dissect is more agressive and it will always print a log on the console.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [May 18, 2022, 11:30am UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/4 "2022-05-18T11:30:20Z")

</div>

Can you provide few lines as sample? If something is classified just replace with dummy data. Need to see fields structure.

---

<div class="post-metadata">

**Author:** ![Thuunder7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thuunder7/32/97482_2.png) [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\
**Post date:** [May 18, 2022, 11:39am UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/5 "2022-05-18T11:39:34Z")

</div>

The pattern that i am using is the following:

```auto
if [metadata][kafka][topic] == "my-log-example"
  dissect { mapping => {
    "message" => "[%{}][%{}][%{[log][type]}]%{}: %{}: %{[log][message]}"
  }

```

I have logs which doesn't have the above pattern , so every time they reach this dissect logstash will print warning logs... I treat these logs later but i would like to silence this warning logs from logstash.

Example of a log without the correct pattern:

```auto
[][evtlog][warn] Queue overflow: 310 events lost

```

Example of a log with correct pattern:

```auto
[xxxxxx][xxxxxx][log][notice] mpgw(xxxxxxxxxxxxx): trans(11111111)[response][x.x.x.x] gtid(1111111111111111): more_info_here

```

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [May 18, 2022, 12:08pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/6 "2022-05-18T12:08:17Z")

</div>

Yes, shouldn't be problem to implement with grok, multi match.

```auto
      grok {
        patterns_dir => "./patterns"
        match => {"message" => ["%{PATTERN1}", "%{PATTERN2}"] }
      } 

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [May 18, 2022, 1:08pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/7 "2022-05-18T13:08:09Z")

</div>

With grok you can have multiple patterns to match in the same filter.

With dissect you can only have one pattern per filter, so the best approach is to filter the type of message and direct it to the correct dissect, you don't need a specific field for this, but you need some pattern in a field.

For example, you shared two types of message, if the one that does not match your pattern always have this string _Queue overflow_, you would be able to filter it out with the following:

```auto
if "Queue overflow" in [message] { do something }

```

But if your only issue is silencing the Dissect filter, you can change the log level.

Run the following in your logstash server and it will stop logging the dissect failure warnings, will log only if the filter has an error.

```auto
curl -XPUT 'localhost:9600/_node/logging?pretty' -H 'Content-Type: application/json' -d'{ "logger.org.logstash.dissect.Dissector" : "ERROR" }'

```

---

<div class="post-metadata">

**Author:** ![Thuunder7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thuunder7/32/97482_2.png) [@Thuunder7](https://discuss.elastic.co/u/Thuunder7)\
**Post date:** [May 18, 2022, 3:59pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/8 "2022-05-18T15:59:02Z")

</div>

I have tried to use the first approach:

```auto
if "Queue overflow" in [message] { do something }

```

And it works perfectly.

Thank you Leandro and Rios

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 15, 2022, 3:59pm UTC](https://discuss.elastic.co/t/best-way-to-parse-multiple-message-patterns/304836/9 "2022-06-15T15:59:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.