# Logstash Grok Issue with Multi-Line Events

**URL:** <https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756>\
**Category:** Logstash\
**Created:** [March 25, 2019, 1:12pm UTC](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756 "2019-03-25T13:12:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![cawoodm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cawoodm/32/14083_2.png) [@cawoodm](https://discuss.elastic.co/u/cawoodm)\
**Post date:** [March 25, 2019, 1:12pm UTC](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756/1 "2019-03-25T13:12:27Z")

</div>

We have the following multi-line log file:

```auto
INFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | ERROR [aaa] 200] [bbb] foo
INFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | Mar 05, 2019 2:54:39 PM bar

```

FileBeat correctly generates the following multi-line message:

```auto
INFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | ERROR [aaa] 200] [bbb] foo
INFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | Mar 05, 2019 2:54:39 PM bar

```

LogStash is now supposed to get the text "ERROR" into the field "type" but misses "ERROR" and finds "Mar".

LogStash Filter:

```auto
grok { 
	match => { "message" => "(?:.*) (?<logdate>%{YEAR}/%{MONTHNUM2}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) \| %{NOTSPACE:type}:? %{GREEDYDATA:raw}" }
}

```

Logstash Generates:

```auto
       "type" => "Mar",
       "message" => "INFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | ERROR [aaa] [200] [bbb] foo\nINFO | jvm 1 | srvmain | 2019/03/25 15:17:00.333 | Mar 05, 2019 2:54:39 PM bar",
       "tags" => [
          [0] "beats_input_codec_plain_applied"
       ],

```

As you can see the type field is "Mar" (matching the date in the second line of the message) instead of "ERROR" matching the first line of the message.

Note: The grok pattern works in the Grok Debugger tool:  
[https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)

You just have to remove the `|` before ERROR because the debugger won't work with `\|` for some reason.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/e/7e386f826495bfc211436fadef1b9cd413369afd.png)

![image](https://us1.discourse-cdn.com/elastic/original/3X/2/f/2f8e185e855c42496992981fa99010f812674d68.png)

---

<div class="post-metadata">

**Author:** ![cawoodm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cawoodm/32/14083_2.png) [@cawoodm](https://discuss.elastic.co/u/cawoodm)\
**Post date:** [March 29, 2019, 1:47pm UTC](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756/2 "2019-03-29T13:47:14Z")

</div>

BUMP: Still having this frustrating issue:

```auto
INFO | jvm 1 | srvmain | 2019/03/29 02:01:37.697 | ERROR [ajp-bio-8009-exec-3] [6B8F5BD240579BE247A50EE34BB6EC4D] [192.168.3.5] [GlobalControllerExceptionHandler] could not translate value expression 'session.catalogversions'
INFO | jvm 1 | srvmain | 2019/03/29 02:01:37.697 | de.hybris.platform.servicelayer.search.exceptions.FlexibleSearchException: could not translate value expression 'session.catalogversions'
INFO | jvm 1 | srvmain | 2019/03/29 02:01:37.697 | at de.hybris.platform.servicelayer.search.impl.DefaultFlexibleSearchService$2.execute(DefaultFlexibleSearchService.java:416) ~[coreserver.jar:?]
INFO | jvm 1 | srvmain | 2019/03/29 02:01:37.697 | at de.hybris.platform.servicelayer.search.impl.DefaultFlexibleSearchService$2.execute(DefaultFlexibleSearchService.java:1) ~[coreserver.jar:?]
INFO | jvm 1 | srvmain | 2019/03/29 02:01:37.697 | at de.hybris.platform.servicelayer.session.impl.DefaultSessionService.executeInLocalView(DefaultSessionService.java:89) ~[coreserver.jar:?]
INFO | jvm 1 | srvmain | 2019/03/29 02:01:37.697 | at de.hybris.platform.servicelayer.search.impl.DefaultFlexibleSearchService.getJaloResult(DefaultFlexibleSearchService.java:395) ~[coreserver.jar:?]
INFO | jvm 1 | srvmain | 2019/03/29 02:01:37.697 | at de.hybris.platform.servicelayer.search.impl.DefaultFlexibleSearchService.search(DefaultFlexibleSearchService.java:167) ~[coreserver.jar:?]

```

Grok:

```auto
grok {
  match => {
    "message" =>"(?:.*) (?<logdate>%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) \|\s+(?<type>[a-zA-Z\-<>]+):? %{GREEDYDATA:raw}"
  }
}

```

Results in `type="at"` instead of `type="ERROR"`.

Here: [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/) it works correctly

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [March 29, 2019, 4:35pm UTC](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756/3 "2019-03-29T16:35:01Z")

</div>

You have issues with tabs vs. spaces in your examples, but try

```
"message" => "(?<logdate>%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) \|\s+(?<type>[a-zA-Z\-<>]+):? %{GREEDYDATA:raw}"

```

i.e., just get rid of the leading (?:.\*)

---

<div class="post-metadata">

**Author:** ![cawoodm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cawoodm/32/14083_2.png) [@cawoodm](https://discuss.elastic.co/u/cawoodm)\
**Post date:** [April 1, 2019, 1:46pm UTC](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756/4 "2019-04-01T13:46:56Z")

</div>

> [@Badger](#):
>
> (?\<logdate\>%{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) |\s+(?\<type\>[a-zA-Z-\<\>]+):? %{GREEDYDATA:raw}

Not sure what you mean: my logs have tabs yes but I use `\s` to deal with that. I've removed the prefix as you said: perhaps that was being too "greedy". We'll watch it and see...

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 29, 2019, 1:47pm UTC](https://discuss.elastic.co/t/logstash-grok-issue-with-multi-line-events/173756/5 "2019-04-29T13:47:03Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.