# Filebeat Multiline - requires pattern to repeat?

**URL:** <https://discuss.elastic.co/t/filebeat-multiline-requires-pattern-to-repeat/277982>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 6, 2021, 3:44pm UTC](https://discuss.elastic.co/t/filebeat-multiline-requires-pattern-to-repeat/277982 "2021-07-06T15:44:43Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![MrDecisive](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mrdecisive/32/78636_2.png) [@MrDecisive](https://discuss.elastic.co/u/MrDecisive)\
**Post date:** [July 6, 2021, 3:44pm UTC](https://discuss.elastic.co/t/filebeat-multiline-requires-pattern-to-repeat/277982/1 "2021-07-06T15:44:43Z")

</div>

I have a file like this.

```auto
[2021-04-22T13:10:06.549Z] Start new event
[2021-04-22T13:10:06.549Z] + set
[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401
[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop
[2021-04-22T13:10:06.549Z] STAGE_NAME='Build Stage'
[2021-04-22T13:10:06.549Z] + ./scripts/make-application.sh
[2021-04-22T13:10:06.549Z] End event
[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event
[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event
[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event
[2021-04-22T13:10:06.549Z] Start new event
[2021-04-22T13:10:06.549Z] + set
[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401
[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop
[2021-04-22T13:10:06.549Z] STAGE_NAME='Test Stage'
[2021-04-22T13:10:06.549Z] + ./scripts/blahblah.sh
[2021-04-22T13:10:06.549Z] End event

```

and a pattern section in the filebeat.yml file like this

```auto
  multiline.pattern: 'Start new event'
  multiline.negate: true
  multiline.match: after
  multiline.flush_pattern: 'End event'

```

and for output I get three event messages

```auto
"message": "[2021-04-22T13:10:06.549Z] Start new event\n[2021-04-22T13:10:06.549Z] + set\n[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401\n[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop\n[2021-04-22T13:10:06.549Z] STAGE_NAME='Build Stage'\n[2021-04-22T13:10:06.549Z] + ./scripts/make-application.sh\n[2021-04-22T13:10:06.549Z] End event"

```

```auto
"message": "[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event\n[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event\n[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event"

```

```auto
"message": "[2021-04-22T13:10:06.549Z] Start new event\n[2021-04-22T13:10:06.549Z] + set\n[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401\n[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop\n[2021-04-22T13:10:06.549Z] STAGE_NAME='Test Stage'\n[2021-04-22T13:10:06.549Z] + ./scripts/blahblah.sh\n[2021-04-22T13:10:06.549Z] End event"

```

but this is not what I want. I want the first message and the third message concatenated together but the lines in between I want to remain as one line, one event??

I've tried all combinations of match and negate settings but none of them work. The documentation gives an example of concatenating a Java exception so that suggests it should work on files where there are some exceptions in amongst other lines but in my case above it does not work...?

I think I need it to work like this -

If (match start event) then {add lines} until {end event} else {output line}

but can't figure out how to make it do this if it is even possible

Any ideas before I give up?

Cheers

---

<div class="post-metadata">

**Author:** ![robertashok](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/robertashok/32/90515_2.png) [@robertashok](https://discuss.elastic.co/u/robertashok)\
**Post date:** [July 7, 2021, 11:30am UTC](https://discuss.elastic.co/t/filebeat-multiline-requires-pattern-to-repeat/277982/2 "2021-07-07T11:30:07Z")

</div>

I think you can use grok in ingest pipeline,

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 4, 2021, 1:30pm UTC](https://discuss.elastic.co/t/filebeat-multiline-requires-pattern-to-repeat/277982/3 "2021-08-04T13:30:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.