# Filebeat exclude lines with multiline

**URL:** <https://discuss.elastic.co/t/filebeat-exclude-lines-with-multiline/161705>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 20, 2018, 2:31pm UTC](https://discuss.elastic.co/t/filebeat-exclude-lines-with-multiline/161705 "2018-12-20T14:31:10Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![swright-eti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/swright-eti/32/38743_2.png) [@swright-eti](https://discuss.elastic.co/u/swright-eti)\
**Post date:** [December 20, 2018, 2:31pm UTC](https://discuss.elastic.co/t/filebeat-exclude-lines-with-multiline/161705/1 "2018-12-20T14:31:10Z")

</div>

I can only seem to get exlude\_lines to work, if multiline is not enabled. I suspect that this due to the order in which these directives are processed. My guess is that the multiline is processed first, which would then make the exclude\_lines not have a match to work with. Can anyone confirm that, or show me how my config is wrong?

Config

```
- type: log
  enabled: true
  paths:
     - /home/eti/logtest/triad-current-msg-format.log
  fields:
        log_type: triad-current-msg
  # Exlude the line of dashes TODO this seems to get ignored when multiline is working. Probably need to strip it in logstash
  exclude_lines: ['^-+$']
  # Setup the pattern to harvest the multiline
  multiline.pattern: '^[A-Z]+: '
  multiline.negate: true
  multiline.match: after

```

Log Pattern

```
ERROR: 12/19/18 02:16:00.225 PID=3126 (cbppvd 1000)
Database Error: Function=add_package Stmt=insert ppvpacks in cborg2001, pack_event_nbr=198740 event_nbr=234824
Code -691: Missing key in referenced table for referential constraint (root.r211_1274).
ISAM Code -111: ISAM error: no record found.
------------------------------------------------------------------------------

```

Thanks.

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [December 20, 2018, 6:09pm UTC](https://discuss.elastic.co/t/filebeat-exclude-lines-with-multiline/161705/2 "2018-12-20T18:09:53Z")

</div>

Hi @swright-eti,

Yes, as you suppose the multiline is processed first, so the problem is that the line with the dashes is considered part of the previous multiline, so it doesn't match. I guess that in your example you see that the last two lines are sent in the same event:

```auto
ISAM Code -111: ISAM error: no record found.
------------------------------------------------------------------------------

```

If your logs always start with `[A-Z]+:` one thing you can try is to add a pattern to the multiline so a line with dashes is considered its own multiline event, then it should be excluded by `exclude_lines`. Something like this:

```auto
multiline.pattern: '(^[A-Z]+: |^-+$)'

```

---

<div class="post-metadata">

**Author:** ![swright-eti](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/swright-eti/32/38743_2.png) [@swright-eti](https://discuss.elastic.co/u/swright-eti)\
**Post date:** [December 20, 2018, 6:17pm UTC](https://discuss.elastic.co/t/filebeat-exclude-lines-with-multiline/161705/3 "2018-12-20T18:17:04Z")

</div>

Thanks. I ended up using mutate in Logstash config.

```auto
mutate {
    gsub => [
        # Replace the line of dashes
        "message", "-+$", ""
    ]
}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 17, 2019, 6:17pm UTC](https://discuss.elastic.co/t/filebeat-exclude-lines-with-multiline/161705/4 "2019-01-17T18:17:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.