# Multiline parsing patterns

**URL:** <https://discuss.elastic.co/t/multiline-parsing-patterns/147171>\
**Category:** Elasticsearch\
**Created:** [September 4, 2018, 8:46am UTC](https://discuss.elastic.co/t/multiline-parsing-patterns/147171 "2018-09-04T08:46:16Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\
**Post date:** [September 4, 2018, 8:46am UTC](https://discuss.elastic.co/t/multiline-parsing-patterns/147171/1 "2018-09-04T08:46:16Z")

</div>

I want to parse a standard JAVA exception which looks like :

```
2018-09-04 05:29:03.955 [default task-38] ERROR c.r.e.u.util.MongoConnectionUtil.createMongoUser - Exception occured while creating mongo userCommand failed with error 11000: 'User "asdf" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
com.mongodb.MongoCommandException: Command failed with error 11000: 'User "qwer" already exists' on server 192.168.1.33:27017. The full response is { "ok" : 0.0, "errmsg" : "User \"asdf\" already exists", "code" : 11000, "codeName" : "DuplicateKey" }
	at com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:115)
	at com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:114)

```

My filebeat.yml has the configuration :

```
filebeat.inputs:

- type: log
  enabled: true
  paths:
    - C:\logs\test.log
  multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
  multiline.negate: false
  multiline.match: after

```

and my logstash.conf input looks like :

```
input {

beats {
	port=>5044
        codec => multiline {
               pattern => "^\s"
              what => "previous"
}
}

```

But logstash says failed to parse the pattern. If I remove the `codec` configuration then only the first line of the exception is getting parsed. Kindly help.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 4, 2018, 3:49pm UTC](https://discuss.elastic.co/t/multiline-parsing-patterns/147171/2 "2018-09-04T15:49:50Z")

</div>

What do you want to aggregate in Logstash? Filebeat already takes care of aggregating multiple lines into a single event. There is no need for further aggregation.

---

<div class="post-metadata">

**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\
**Post date:** [September 6, 2018, 3:47am UTC](https://discuss.elastic.co/t/multiline-parsing-patterns/147171/3 "2018-09-06T03:47:43Z")

</div>

ok, so let say I removed the logstash `codec` configuration. So whatever I posted for filebeat is that enough for aggregation of logs for JAVA exception stackstrace ? Is the configuration correct ? Because I don't see it happening.

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [September 6, 2018, 7:18am UTC](https://discuss.elastic.co/t/multiline-parsing-patterns/147171/4 "2018-09-06T07:18:51Z")

</div>

Yes, the log you've pasted here differs from the examples here: [https://www.elastic.co/guide/en/beats/filebeat/current/\_examples\_of\_multiline\_configuration.html#\_java\_stack\_traces](https://www.elastic.co/guide/en/beats/filebeat/current/_examples_of_multiline_configuration.html#_java_stack_traces)

You could try to match for the beginning of the logs which is a timestamp.

```auto
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after

```

---

<div class="post-metadata">

**Author:** ![Raghuveer\_SJ](https://avatars.discourse-cdn.com/v4/letter/r/2bfe46/32.png) [@Raghuveer\_SJ](https://discuss.elastic.co/u/Raghuveer_SJ)\
**Post date:** [September 7, 2018, 9:41am UTC](https://discuss.elastic.co/t/multiline-parsing-patterns/147171/5 "2018-09-07T09:41:57Z")

</div>

So this means log starting with a timestamp and anything after that should be grouped till you see a log with another timestamp right ? (I am pointing to `caused by` text that will be there in exceptions)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 5, 2018, 9:41am UTC](https://discuss.elastic.co/t/multiline-parsing-patterns/147171/6 "2018-10-05T09:41:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.