Parsing multiline, look for closing character?

Elastic Stack Logstash
1

I am working on a project to parse vCenter logs and when vCenter passes an Event, it puts the event details on multiple messages(see below).

"message":"<14>1 2018-02-14T10:01:12.888790-06:00 vcenter01 vpxd 4683 - - Event [96904] [1-1] [2018-02-14T16:01:12.888057Z] [vim.event.VmReconfiguredEvent] [info] [VSPHERE.LOCAL\Administrator] [DC] [96903] [Reconfigured eventtest on esxi01.domain.com in DC. "
"message":" "
"message":"Modified: "
"message":" "
"message":"config.hardware.numCPU: 2 -> 1; "
"message":""
"message":" Added: "
"message":" "
"message":" Deleted: "
"message":" "
"message":"]"

Is it possible to have logstash, when it encounters the closing "]" that it will search the above messages for the opening and concat it into one? Or is there a better solution to combine these messages into one?

2

Yes, a multiline codec will do that.

    codec => multiline {
      what => "next"
      negate => "true"
      pattern => '"message":"]"'
    }
3

Thanks for the reply. My understanding might be lacking but how does it know which lines to concat or where to start with this codec?

4

The documentation is here. That combination of what and negate mean if the line does not match the pattern, then prepend this line to the next line. An event is not emitted until a line does match the pattern.

5

I figured out a slightly different route to go. The messages that i receive all start with so i setup the multiline codec to postpend any line that doesn't start with that pattern. Code below.

codec => multiline {
pattern => "^<%{POSINT}>1 "
negate => true
what => "previous"
}