Multiline matching keeps failing


(Robert) #1

I've been going at this for weeks now and I can't seem to wrap my head around what's wrong about this.

I'm trying to get all of these lines to fit into a multiline match with grok, but it only picks up the last one, and even discards the digit at the beginning of the line.

11:31:03.936	<           :     1>	5: Load times per type (ms):
12: aaaaaa.aaaaaaaaa.aaaaaaa.aaaaaaa
1: bbbb.bbbb.bbbbbbbbbbbbb.bbbbbbbbb
3: cccc.cccccccc.ccccccccccc.cccccc
64: ddd.dddddddddddd.ddddddd.ddddddd

Expected result:

message_processed = Load times per type (ms):
12: aaaaaa.aaaaaaaaa.aaaaaaa.aaaaaaa
1: bbbb.bbbb.bbbbbbbbbbbbb.bbbbbbbbb
3: cccc.cccccccc.ccccccccccc.cccccc
64: ddd.dddddddddddd.ddddddd.ddddddd

Actual result:

message_processed = ddd.dddddddddddd.ddddddd.ddddddd

I'm using the following grok pattern:

grok {
        match => [ "message" , "%{TIME:time}.*%{NUMBER:loglevel}:\s%{GREEDYDATA:message_processed}" ]
    }

It is being shipped to logstash with filebeat on a windows server with the following multi-line config in filebeat.yml:

multiline.pattern: ^[0-9]{2}\:[0-9]{2}\:[0-9]{2}
multiline.negate: true
multiline.match: after

I've tried using (?m) flag but to no avail, and I also tried using (?<message_processed>(.|\r|\n)*) instead of %{GREEDYDATA:message_processed} but that caused the logs to not even load at all, despite having checked that config validation is OK.
The pattern correctly picks up all lines with grokconstructor, but when I run my logs through, it keeps failing.

What am I doing wrong?


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.