Display log in a given format

Elastic Stack Logstash
1

Hello,

I put the below code to group lines which start with "START" tag:

codec => multiline {
pattern => "^START"
negate => true
what => previous
auto_flush_interval => 1
}

but i want to group line only which comes between each start and end. For example:

START
INFO: NOTIFICATION_ID,NOTIFICATION_TYPE
Web Serivce call
END
unwanted text
START
INFO: NOTIFICATION_order
Web Serivce another call
END

In the above example i want to remove the "unwanted text" line because it doesn't come between "START" and "END"
The output should be like:

Message 1:
NFO: NOTIFICATION_ID,NOTIFICATION_TYPE
Web Serivce call

Message 2:
INFO: NOTIFICATION_order
Web Serivce another call

Request you to please help me.

Regards,
Deepika

2

Perhaps you can use your current multiline configuration but use a grok filter or a mutate filter's gsub option to remove "\nEND\n" and everything that follows?

3

Thanks !!
But i am not able to use grok .
Everytime it gives the error. can you pls give me one example

4

There are thousands of grok examples out there, including in the plugin's documentation. But I actually think gsub is better in this case, so you can try

mutate {
  gsub => ["message", "\nEND\n.*", ""]
}

but I suspect that it won't work since \n isn't properly interpreted as a newline character. Using \s should work but could lead to false positives since it matches all whitespace characters.