# Filebeat multiline setting not working

**URL:** <https://discuss.elastic.co/t/filebeat-multiline-setting-not-working/194263>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [August 7, 2019, 2:54pm UTC](https://discuss.elastic.co/t/filebeat-multiline-setting-not-working/194263 "2019-08-07T14:54:37Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![rituzy](https://avatars.discourse-cdn.com/v4/letter/r/e274bd/32.png) [@rituzy](https://discuss.elastic.co/u/rituzy)\
**Post date:** [August 7, 2019, 2:54pm UTC](https://discuss.elastic.co/t/filebeat-multiline-setting-not-working/194263/1 "2019-08-07T14:54:37Z")

</div>

Hi, I was trying to consolidate stack trace by filebeat as per instruction here

> **[Manage multiline messages | Filebeat Reference \[7.0\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.0/multiline-examples.html)**

This is my filebeat.yml settings

```
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /home/myHomeFolder/log_example.log    
  fields:
    app: myapp
    multiline.pattern: '^Excep'
    multiline.negate: true
    multiline.match: after

  name: filebeat-myapp-test

tags: ["", "env-test"]

# Optional fields that you can specify to add additional information to the
# output.
fields:
  env: test
  app: myapp

```

This is an example log file /home/myHomeFolder/log\_example.log :

> ```
> Exception in thread "main" java.lang.NullPointerException
> at com.example.myproject.Book.getTitle(Book.java:16)
> at com.example.myproject.Author.getBookTitles(Author.java:25)
> at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
> Exception in thread "main" java.lang.NullPointerException
> at com.example.myproject.Book.getTitle(Book.java:16)
> at com.example.myproject.Author.getBookTitles(Author.java:25)
> at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
> Exception in thread "main" java.lang.NullPointerException
> at com.example.myproject.Book.getTitle(Book.java:16)
> at com.example.myproject.Author.getBookTitles(Author.java:25)
> at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
> Exception in thread "main" java.lang.NullPointerException
> at com.example.myproject.Book.getTitle(Book.java:16)
> at com.example.myproject.Author.getBookTitles(Author.java:25)
> at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
> Exception in thread "main" java.lang.NullPointerException
> at com.example.myproject.Book.getTitle(Book.java:16)
> at com.example.myproject.Author.getBookTitles(Author.java:25)
> at com.example.myproject.Bootstrap.main(Bootstrap.java:14)
> 
> ```

This is an output of command  
`sudo /usr/share/filebeat/bin/filebeat -e -c /home/myHomeFolder/filebeat.yml`  
(part of the output) :

> . . .  
> {  
> "@timestamp": "2019-08-07T14:29:13.559Z",  
> "@metadata": {  
> "beat": "filebeat",  
> "type": "\_doc",  
> "version": "7.0.1"  
> },  
> "log": {  
> "file": {  
> "path": "/home/myHomeFolder/log\_example.log"  
> },  
> "offset": 2821  
> },  
> "message": "Exception in thread "main" java.lang.NullPointerException",  
> "tags": [  
> "",  
> "env-test"  
> ],  
> "input": {  
> "type": "log"  
> },  
> "fields": {  
> "env": "test",  
> "app": "myapp",  
> "multiline": {  
> "pattern": "^Excep",  
> "negate": true,  
> "match": "after"  
> }  
> },  
> "ecs": {  
> "version": "1.0.0"  
> },  
> "host": {  
> "name": "myapp-test"  
> },  
> "agent": {  
> "id": "0f1992b2-2dce-486c-a400-76cbe7e16a06",  
> "version": "7.0.1",  
> "type": "filebeat",  
> "ephemeral\_id": "6a1ef472-992a-40ff-a448-04fa037e3bef",  
> "hostname": "myapp-test"  
> }  
> }  
> {  
> "@timestamp": "2019-08-07T14:29:13.559Z",  
> "@metadata": {  
> "beat": "filebeat",  
> "type": "\_doc",  
> "version": "7.0.1"  
> },  
> "message": " at com.example.myproject.Book.getTitle(Book.java:16)",  
> "tags": [  
> "",  
> "env-test"  
> ],  
> "input": {  
> "type": "log"  
> },  
> "fields": {  
> "env": "test",  
> "app": "myapp",  
> "multiline": {  
> "pattern": "^Excep",  
> "negate": true,  
> "match": "after"  
> }  
> },  
> "agent": {  
> "id": "0f1992b2-2dce-486c-a400-76cbe7e16a06",  
> "version": "7.0.1",  
> "type": "filebeat",  
> "ephemeral\_id": "6a1ef472-992a-40ff-a448-04fa037e3bef",  
> "hostname": "myapp-test"  
> },  
> "ecs": {  
> "version": "1.0.0"  
> },  
> "host": {  
> "name": "myapp-test"  
> },  
> "log": {  
> "offset": 2879,  
> "file": {  
> "path": "/home/myHomeFolder/log\_example.log"  
> }  
> }  
> }  
> {  
> "@timestamp": "2019-08-07T14:29:13.559Z",  
> "@metadata": {  
> "beat": "filebeat",  
> "type": "\_doc",  
> "version": "7.0.1"  
> },  
> "message": " at com.example.myproject.Author.getBookTitles(Author.java:25)",  
> . . .

As you can see, it splits every message separately(every line one by one) But I want to gather all this staff for the stack trace in a one message and than send to logstash server.

I tried to switch multiline.negate to true or false and no luck

Can anyone help with this?

There is one error on filebeat starting  
`2019-08-07T17:28:23.554+0300 ERROR fileset/modules.go:125 Not loading modules. Module directory not found: /usr/share/filebeat/bin/module`  
and one warning  
`2019-08-07T17:28:23.555+0300 WARN beater/filebeat.go:357 Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.`  
But I believe it does not affect at all...

Thanx in advance!

---

<div class="post-metadata">

**Author:** ![justkind](https://avatars.discourse-cdn.com/v4/letter/j/919ad9/32.png) [@justkind](https://discuss.elastic.co/u/justkind)\
**Post date:** [August 7, 2019, 9:27pm UTC](https://discuss.elastic.co/t/filebeat-multiline-setting-not-working/194263/2 "2019-08-07T21:27:10Z")

</div>

Can you try moving your multiline options up a level?

I don't think it should be under the fields option as it should be on the same level according to the log input documentation:  
[https://www.elastic.co/guide/en/beats/filebeat/7.0/filebeat-input-log.html](https://www.elastic.co/guide/en/beats/filebeat/7.0/filebeat-input-log.html)

Ex.

```
fields:
  app: myapp
multiline.pattern: '^Excep'
multiline.negate: true
multiline.match: after

```

You can also look at a reference yml here and see how everything is setup:  
[https://www.elastic.co/guide/en/beats/filebeat/7.0/filebeat-reference-yml.html](https://www.elastic.co/guide/en/beats/filebeat/7.0/filebeat-reference-yml.html)

Regarding your multiline pattern, you can test it on the Go Playground, Filebeat has documentation on this at: [https://www.elastic.co/guide/en/beats/filebeat/7.0/\_test\_your\_regexp\_pattern\_for\_multiline.html](https://www.elastic.co/guide/en/beats/filebeat/7.0/_test_your_regexp_pattern_for_multiline.html)

According to the [playground](https://play.golang.org/p/NAjLDgXPN3W), your pattern seems to do what you want it to do so if you can get the multiline options to work everything should fall into place

---

<div class="post-metadata">

**Author:** ![rituzy](https://avatars.discourse-cdn.com/v4/letter/r/e274bd/32.png) [@rituzy](https://discuss.elastic.co/u/rituzy)\
**Post date:** [August 8, 2019, 8:19am UTC](https://discuss.elastic.co/t/filebeat-multiline-setting-not-working/194263/3 "2019-08-08T08:19:01Z")

</div>

Bingo!  
It was the root problem, now it does as expected:

> ```
> {
> "@timestamp": "2019-08-08T07:59:02.414Z",
> "@metadata": {
> "beat": "filebeat",
> "type": "_doc",
> "version": "7.0.1"
> },
> "message": "Exception in thread \"main\" java.lang.NullPointerException\n at com.example.myproject.Book.getTitle(Book.java:16)\n at com.example.myproject.Author.getBookTitles(Author.java:25)\n at com.example.myproject.Bootstrap.main(Bootstrap.java:14)\n",
> "log": {
> "offset": 0,
> "file": {
> "path": "/home/myHomeFolder/log_example.log"
> },
> "flags": [
> "multiline"
> ]
> },
> 
> ```

Thank you so much for you prompt fruitful reply!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 5, 2019, 8:19am UTC](https://discuss.elastic.co/t/filebeat-multiline-setting-not-working/194263/4 "2019-09-05T08:19:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.