# Grok pattern does not work in logstash however it works in kibana

**URL:** <https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148>\
**Category:** Logstash\
**Created:** [September 19, 2019, 8:26am UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148 "2019-09-19T08:26:40Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [September 19, 2019, 8:26am UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/1 "2019-09-19T08:26:41Z")

</div>

Hello,

I have a long used grok pattern match that stopped working since upgraded to ELK 7.3.0.

```
grok {
   match => { "message" => ["%{MONTH:month} %{MONTHDAY:day}, %{YEAR:year} %{TIME:time} %{WORD:day_period} %{NOTSPACE:[system][jetty][class]} %{WORD:[system][jetty][method]}\\n%{GREEDYMULTILINE:multiline}",
                           "%{WORD:severity} in thread \"%{THREADNAME:threadName}\" %{GREEDYDATA:[system][jetty][data]}",
                           "%{WORD:severity} in thread \"%{THREADNAME:threadName}\" %{NOTSPACE:[system][jetty][exceptionClass]}: %{GREEDYDATA:exceptionMessage}\\n %{GREEDYMULTILINE:exceptionMultiline}"
                           ] }
   pattern_definitions => { 
      "GREEDYMULTILINE" => "(.|\r|\n)*",
      "THREADNAME" => "[^\"]+"
   }
   remove_field => "host"
}

```

This grok match does not split the message on fields while using grok pattern matcher from kibana all goes well as it can be seen in the picture uploaded.

 ![16](https://us1.discourse-cdn.com/elastic/original/3X/a/8/a82ba617570c2a19694e0fa3e83f3645dd733f9b.png)

Any suggestion how to fix it?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [September 23, 2019, 4:17pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/2 "2019-09-23T16:17:39Z")

</div>

Any suggestion for this?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 23, 2019, 5:57pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/3 "2019-09-23T17:57:44Z")

</div>

Do you have a correctly configured multiline codec in place?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [September 23, 2019, 5:59pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/4 "2019-09-23T17:59:49Z")

</div>

The codec was working for months on 6.x. Suddenly stopped working when I switched to 7.3.0.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 23, 2019, 6:00pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/5 "2019-09-23T18:00:33Z")

</div>

Then it seems to be the codec that is the issue. How is it configured?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [September 23, 2019, 6:03pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/6 "2019-09-23T18:03:07Z")

</div>

Not sure which codec are we talking about? This is a grok matching filter option which has the possibility to create custom patterns.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 23, 2019, 6:09pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/7 "2019-09-23T18:09:06Z")

</div>

Which inputs are you using? Are the events that are not parsed correctly having multiple lines in the message field?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [September 23, 2019, 6:10pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/8 "2019-09-23T18:10:52Z")

</div>

I am using beats the information is coming from filebeat. The filebeat is configured correctly since the full message is correct. The issue is with the grok custom pattern feature. For some reason the same pattern is splitting the message into fields but when it gets to the grok filter it does not work.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 23, 2019, 6:11pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/9 "2019-09-23T18:11:47Z")

</div>

I am not sure I follow. Can you show anevent that has been indexed and not been parsed correctly?

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [September 23, 2019, 6:15pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/10 "2019-09-23T18:15:59Z")

</div>

Here you have a link to an event that is not splitted into fields.

[https://pastebin.com/z1LRBQPx](https://pastebin.com/z1LRBQPx)

I think it might be the case of the message not interpretting the `\n` while the grok considers that as a word and not a new line.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 23, 2019, 7:06pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/11 "2019-09-23T19:06:13Z")

</div>

> [@zozo6015](#):
>
> pattern\_definitions =\> { "GREEDYMULTILINE" =\> "(.|\r|\n)\*", "THREADNAME" =\> "[^"]+" }

You cannot have a comma between patterns. It will not compile.

If you have a literal newline in your data then use one in the pattern

```
"%{MONTH:month} %{MONTHDAY:day}, %{YEAR:year} %{TIME:time} %{WORD:day_period} %{NOTSPACE:[system][jetty][class]} %{WORD:[system][jetty][method]}
%{GREEDYMULTILINE:multiline}"

```

---

<div class="post-metadata">

**Author:** ![zozo6015](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zozo6015/32/12117_2.png) [@zozo6015](https://discuss.elastic.co/u/zozo6015)\
**Post date:** [September 23, 2019, 7:12pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/12 "2019-09-23T19:12:56Z")

</div>

Changed the config but still not splitting up the message fields on smaller fields.

```
 grok {
   match => { "message" => ["%{MONTH:month} %{MONTHDAY:day}, %{YEAR:year} %{TIME:time} %{WORD:day_period} %{NOTSPACE:[system][jetty][class]} %{WORD:[system][jetty][method]}(:?\n|\\n)%{GREEDYMULTILINE:multiline}",
                           "%{WORD:severity} in thread \"%{THREADNAME:threadName}\" %{GREEDYDATA:[system][jetty][data]}",
                           "%{WORD:severity} in thread \"%{THREADNAME:threadName}\" %{NOTSPACE:[system][jetty][exceptionClass]}: %{GREEDYDATA:exceptionMessage}(:?\n|\\n) %{GREEDYMULTILINE:exceptionMultiline}"
                           ] }
   pattern_definitions => {
     "GREEDYMULTILINE" => "(.|\r|\n)*"
     "THREADNAME" => "[^\"]+"
   }
   remove_field => "host"
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 21, 2019, 7:12pm UTC](https://discuss.elastic.co/t/grok-pattern-does-not-work-in-logstash-however-it-works-in-kibana/200148/13 "2019-10-21T19:12:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.