# Split multiple lines (using Grok filter) in message so they display as newlines in Kibana

**URL:** https://discuss.elastic.co/t/split-multiple-lines-using-grok-filter-in-message-so-they-display-as-newlines-in-kibana/136431
**Category:** Logstash
**Created:** [June 19, 2018, 6:57am UTC](https://discuss.elastic.co/t/split-multiple-lines-using-grok-filter-in-message-so-they-display-as-newlines-in-kibana/136431 "2018-06-19T06:57:33Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![Warren](https://avatars.discourse-cdn.com/v4/letter/w/97f17d/32.png) [@Warren](https://discuss.elastic.co/u/Warren)
#### Post date: [June 19, 2018, 6:57am UTC](https://discuss.elastic.co/t/split-multiple-lines-using-grok-filter-in-message-so-they-display-as-newlines-in-kibana/136431/1 "2018-06-19T06:57:33Z")

</div>

So i have a message in Kibana and it looks like this:

[2018-06-07 11:02:33] testing.ERROR: Class 'App\Http\Controllers\Deal' not found {"exception":"[object] (Symfony\Component\Debug\Exception\FatalThrowableError(code: 0): Class 'App\Http\Controllers\Deal' not found at /app/Http/Controllers/TestingSyslogController.php:13)\n[stacktrace]\n#0 [internal function]: App\Http\Controllers\TestingSyslogController-\>index()\n#1 /vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): call\_user\_func\_array(Array, Array)\n#2 /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\Routing\Controller-\>callAction('index', Array)\n#3 /vendor/laravel/framework/src/Illuminate/Routing/Route.php(212): Illuminate\Routing\ControllerDispatcher-\>dispatch(Object(Illuminate\Routing\Route), Object(App\Http\Controllers\TestingSyslogController), 'index')\n#4 /vendor/laravel/framework/src/Illuminate/Routing/Route.php(169): Illuminate\Routing\Route-\>runController()\n#5

How do I display the message line by line in a single message field e.g. use the \n to display the message in kibana like this

[2018-06-07 11:02:33] testing.ERROR: Class 'App\Http\Controllers\Deal' not found {"exception":"[object] (Symfony\Component\Debug\Exception\FatalThrowableError(code: 0): Class 'App\Http\Controllers\Deal' not found at /app/Http/Controllers/TestingSyslogController.php:13)\n[stacktrace]\n

0 [internal function]: App\Http\Controllers\TestingSyslogController-\>index()\n

1 /vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): call\_user\_func\_array(Array, Array)\n

2 /vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\Routing\Controller-\>callAction('index', Array)\n

3 /vendor/laravel/framework/src/Illuminate/Routing/Route.php(212): Illuminate\Routing\ControllerDispatcher-\>dispatch(Object(Illuminate\Routing\Route), Object(App\Http\Controllers\TestingSyslogController), 'index')\n

4 /vendor/laravel/framework/src/Illuminate/Routing/Route.php(169): Illuminate\Routing\Route-\>runController()\n#5

I'm trying a grok filter... and the debugger at [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/) is very useful.

I've written this:  
%{DATESTAMP:timestamp}%{SPACE}]%{SPACE}%{NOTSPACE:instance}%{SPACE}(?(.|\r|\n)\*)

but it still shows a single line...

This regex would match the line break.. \[a-z]{1}  
and this rexeg should match the line break and line number \[a-z]{1}(?:#)\d{1}

but neither of these match

%{DATESTAMP:timestamp}%{SPACE}]%{SPACE}%{NOTSPACE:instance}%{SPACE}(?\[a-z]1})

%{DATESTAMP:timestamp}%{SPACE}]%{SPACE}%{NOTSPACE:instance}%{SPACE}(?\[a-z]{1}(?:#)\d{1,2})

---

<div class="post-metadata">

### Author: ![Carmageddon](https://avatars.discourse-cdn.com/v4/letter/c/db5fbb/32.png) [@Carmageddon](https://discuss.elastic.co/u/Carmageddon)
#### Post date: [June 21, 2018, 7:05pm UTC](https://discuss.elastic.co/t/split-multiple-lines-using-grok-filter-in-message-so-they-display-as-newlines-in-kibana/136431/2 "2018-06-21T19:05:41Z")

</div>

I've had the exactly same issue you did the last two days.  
Until I stepped back, and thought "why the hell are we still using GROK filter and regex to break it down?"

Turns out it is no longer needed, it was carry over from legacy system that has been decomissioned.  
I tested the same message structured in a JSON format instead - works perfectly no need to break it down into fields using any GROK regex.  
Attached example how it looks like in Kibana:

 ![Kibana%20Multilines](https://us1.discourse-cdn.com/elastic/original/3X/8/5/85b55194ac849bab3c311562140b52c93ed89481.PNG)

JSON looks like:

> {.... ,"failureReason": "java.lang.AssertionError: expected [35] but found [25.]\n\tat org.testng.Assert.fail(Assert.java:94)\n......",}

Please note that I lost 2 hours until I realized each ROW in logfile should be a JSON object {...}.  
You cant send 2+ objects in an array like [{..},{..}] - filebeat cant unmarshal that.

Hope that helps, Good luck!

---

<div class="post-metadata">

### Author: ![Warren](https://avatars.discourse-cdn.com/v4/letter/w/97f17d/32.png) [@Warren](https://discuss.elastic.co/u/Warren)
#### Post date: [June 22, 2018, 10:53am UTC](https://discuss.elastic.co/t/split-multiple-lines-using-grok-filter-in-message-so-they-display-as-newlines-in-kibana/136431/3 "2018-06-22T10:53:47Z")

</div>

Hi,  
Thanks for replying.  
So I don't need this startmsg.regex in my rsyslog conf file? but then they will be all separate rsyslog entries? Do I need to use multiline filter?

input(type="imfile"  
File="/home/app/2018-06-22.log"  
startmsg.regex="[[:digit:]]{4}-[[:digit:]]{1,2}-[[:digit:]]{1,2} [[:digit:]]{1,2}:[[:digit:]]{1,2}:[[:digit:]]{1,2}"  
)

This is my current Rsyslog JSON template...

template(name="json-template"  
type="list") {  
constant(value="{")  
constant(value=""@timestamp":"") property(name="timereported" dateFormat="rfc3339")  
constant(value="","@version":"1")  
constant(value="","message":"") property(name="msg" format="json")  
constant(value="","sysloghost":"") property(name="hostname")  
constant(value="","severity":"") property(name="syslogseverity-text")  
constant(value="","facility":"") property(name="syslogfacility-text")  
constant(value="","programname":"") property(name="programname")  
constant(value="","procid":"") property(name="procid")  
constant(value=""}\n")  
}

---

<div class="post-metadata">

### Author: ![Carmageddon](https://avatars.discourse-cdn.com/v4/letter/c/db5fbb/32.png) [@Carmageddon](https://discuss.elastic.co/u/Carmageddon)
#### Post date: [June 22, 2018, 12:41pm UTC](https://discuss.elastic.co/t/split-multiple-lines-using-grok-filter-in-message-so-they-display-as-newlines-in-kibana/136431/4 "2018-06-22T12:41:16Z")

</div>

Let me clarify - the input log is controlled by me, so I am easily able to modify it to output JSON instead of the log line like yours, and this solves the problem for me. I dont know anything about Rsyslog configuration.

If you cant modify your input to place JSON, perhaps you can manipulate the input and write a JSON into another file, and only monitor/submit to Logstash only that processed file? You wont have to deal with GROK then.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 20, 2018, 12:41pm UTC](https://discuss.elastic.co/t/split-multiple-lines-using-grok-filter-in-message-so-they-display-as-newlines-in-kibana/136431/5 "2018-07-20T12:41:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
