# What does filebeat's multiline tool match

**URL:** <https://discuss.elastic.co/t/what-does-filebeats-multiline-tool-match/225720>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [March 30, 2020, 5:25pm UTC](https://discuss.elastic.co/t/what-does-filebeats-multiline-tool-match/225720 "2020-03-30T17:25:29Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![kevin5617](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@kevin5617](https://discuss.elastic.co/u/kevin5617)\
**Post date:** [March 30, 2020, 5:25pm UTC](https://discuss.elastic.co/t/what-does-filebeats-multiline-tool-match/225720/1 "2020-03-30T17:25:30Z")

</div>

I am using filebeat to read Docker logs and feed them to logstash. I have some logs that are being split into separate events. This happens with stack traces, or just any logs with a new line in them.

Here is my filebeat.yml file.

```auto
    filebeat.autodiscover:
        providers:
            - type: docker
                templates:
                    - config:
                        - type: container
                            paths:
                                - "/var/lib/docker/containers/${data.docker.container.id}/*.log"
    logging.metrics.enabled: false
    output.logstash:
        hosts:
            - logstash:5044

```

I have a couple questions about this. First off, if I look in those `.log` files, they are in the following format

```auto
    {"log":"Message is in here\n", "stream":"stderr", "time":"<timestamp here>"}
    {"log":"\u0009this line should be added to the previous log\n", "stream":"stderr", "time":"<timestamp here>"}

```

In the multiline.pattern field, is it looking to match the log part only, or the entire line starting with the {? Basically it comes down to should I use

```
multiline.pattern: '^{"log":"\t'

```

or

```
multiline.pattern: '^\t'

```

In addition, will this tool even work because i just want to take the log part and append it to the previous log message?

---

<div class="post-metadata">

**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\
**Post date:** [March 31, 2020, 8:16am UTC](https://discuss.elastic.co/t/what-does-filebeats-multiline-tool-match/225720/2 "2020-03-31T08:16:15Z")

</div>

Hi!

Since you want to handle json logs I would suggest you playing around with the json related features that Filebeat provides:

[https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-config-json](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-log.html#filebeat-input-log-config-json)  
[https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html](https://www.elastic.co/guide/en/beats/filebeat/current/decode-json-fields.html)

---

<div class="post-metadata">

**Author:** ![kevin5617](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@kevin5617](https://discuss.elastic.co/u/kevin5617)\
**Post date:** [March 31, 2020, 6:00pm UTC](https://discuss.elastic.co/t/what-does-filebeats-multiline-tool-match/225720/3 "2020-03-31T18:00:48Z")

</div>

> [@kevin5617](#):
>
> filebeat.autodiscover: providers: - type: docker templates: - config: - type: container paths: - "/var/lib/docker/containers/${data.docker.container.id}/\*.log" logging.metrics.enabled: false output.logstash: hosts: - logstash:5044

Hi so this is now my filebeat.yml. Does this look roughly right to you?

```auto
filebeat.autodiscover:
    providers:
        - type: docker
            templates:
                - config:
                    - type: container
                        paths:
                            - "/var/lib/docker/containers/${data.docker.container.id}/*.log"
            json.message_key: log
            multiline.pattern: ^\t
            multiline.negate: false
            multiline.match: after
logging.metrics.enabled: false
output.logstash:
    hosts:
        - logstash:5044

```

---

<div class="post-metadata">

**Author:** ![kevin5617](https://avatars.discourse-cdn.com/v4/letter/k/46a35a/32.png) [@kevin5617](https://discuss.elastic.co/u/kevin5617)\
**Post date:** [March 31, 2020, 9:07pm UTC](https://discuss.elastic.co/t/what-does-filebeats-multiline-tool-match/225720/4 "2020-03-31T21:07:22Z")

</div>

I found the solution for my case. This is now my filebeat.yml file

```auto
filebeat.autodiscover:
        providers:
            - type: docker
                templates:
                    - config:
                        - type: container
                            paths:
                                - "/var/lib/docker/containers/${data.docker.container.id}/*.log"
                        multiline.pattern: "^\t"
                        multiline.negate: false
                        multiline.match: after
    logging.metrics.enabled: false
    output.logstash:
        hosts:
            - logstash:5044

```

I had the three multiline lines at the wrong "level". Originally they were right between the `- type: docker` and `templates:` lines.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 28, 2020, 9:07pm UTC](https://discuss.elastic.co/t/what-does-filebeats-multiline-tool-match/225720/5 "2020-04-28T21:07:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.