Filebeat Multiline - requires pattern to repeat?

I have a file like this.

[2021-04-22T13:10:06.549Z] Start new event
[2021-04-22T13:10:06.549Z] + set
[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401
[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop
[2021-04-22T13:10:06.549Z] STAGE_NAME='Build Stage'
[2021-04-22T13:10:06.549Z] + ./scripts/make-application.sh
[2021-04-22T13:10:06.549Z] End event
[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event
[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event
[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event
[2021-04-22T13:10:06.549Z] Start new event
[2021-04-22T13:10:06.549Z] + set
[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401
[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop
[2021-04-22T13:10:06.549Z] STAGE_NAME='Test Stage'
[2021-04-22T13:10:06.549Z] + ./scripts/blahblah.sh
[2021-04-22T13:10:06.549Z] End event

and a pattern section in the filebeat.yml file like this

  multiline.pattern: 'Start new event'
  multiline.negate: true
  multiline.match: after
  multiline.flush_pattern: 'End event'

and for output I get three event messages

"message": "[2021-04-22T13:10:06.549Z] Start new event\n[2021-04-22T13:10:06.549Z] + set\n[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401\n[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop\n[2021-04-22T13:10:06.549Z] STAGE_NAME='Build Stage'\n[2021-04-22T13:10:06.549Z] + ./scripts/make-application.sh\n[2021-04-22T13:10:06.549Z] End event"
"message": "[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event\n[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event\n[2021-04-22T13:10:06.549Z] Some other line which I want to be a single line event"
"message": "[2021-04-22T13:10:06.549Z] Start new event\n[2021-04-22T13:10:06.549Z] + set\n[2021-04-22T13:10:06.549Z] BUILD_NUMBER=401\n[2021-04-22T13:10:06.549Z] BRANCH_NAME=develop\n[2021-04-22T13:10:06.549Z] STAGE_NAME='Test Stage'\n[2021-04-22T13:10:06.549Z] + ./scripts/blahblah.sh\n[2021-04-22T13:10:06.549Z] End event"

but this is not what I want. I want the first message and the third message concatenated together but the lines in between I want to remain as one line, one event??

I've tried all combinations of match and negate settings but none of them work. The documentation gives an example of concatenating a Java exception so that suggests it should work on files where there are some exceptions in amongst other lines but in my case above it does not work...?

I think I need it to work like this -

If (match start event) then {add lines} until {end event} else {output line}

but can't figure out how to make it do this if it is even possible

Any ideas before I give up?

Cheers

I think you can use grok in ingest pipeline,