Multi-Line Codec Help

[cawoodm (Marc)]

We need some help with the multiline codec.

We have syslog entries arriving - the message lines look like this:

jvm 1   : ERROR [fooHTTP38] [10.1.102.49] [AbstractResource] Error crea
jvm 1   : de.foo.platform.webservices.BadRequestException: Error creati
jvm 1   : at de.foo.platform.webservices.HttpPutResponseBuilder.cre
jvm 1   : Caused by: de.foo.platform.servicelayer.exceptions.ModelSav
jvm 1   : 	at de.foo.platform.servicelayer.internal.model.impl.wrapp
jvm 1   : 	at de.foo.platform.servicelayer.internal.model.impl.wrapp
jvm 1   : 	at de.foo.platform.servicelayer.internal.model.extractor.
jvm 1   : 	at de.foo.platform.servicelayer.internal.model.impl.Defau
jvm 1   : 	at de.foo.platform.servicelayer.internal.model.impl.Defau
jvm 1   : 	at de.foo.platform.servicelayer.internal.model.impl.Defau
jvm 1   : 	at de.foo.platform.servicelayer.internal.model.impl.Defau
jvm 1   : 	at de.foo.platform.webservices.AbstractYResource.createOr
jvm 1   : 	at de.foo.platform.webservices.HttpPutResponseBuilder.cre
jvm 1   : 	at de.foo.platform.webservices.HttpPutResponseBuilder.cre
jvm 1   : 	... 101 more

Each line is logged separately as "INFO" but actually this is one event of type ERROR - the first line is the event and the other lines are multi-lines belonging to the event.

Our first challenge is to parse this with the multiline codec. We have seen that all events start with either ERROR, WARN, INFO, DEBUG etc. We have tried the following in our syslog input section:

codec => multiline {
    #Every line which doesn't contain this is a continuation of the previous event:
     pattern => "(SEVERE)|(ERROR)|(WARN)|(INFO)|(DEBUG)"
     negate => true
    what => "previous"
}

This does not work - we get no events in our output although the RegEx is apparently valid.

[Dvikas (Vikas)]

multiline features u can do in filebeat,if you are using it.for eg:

Multiline options

Mutiline can be used for log messages spanning multiple lines. This is common

for Java Stack Traces or C-Line Continuation

The regexp Pattern that has to be matched. The example pattern matches all lines starting with [

multiline.pattern: provide the start of the line.I mean if your line starts with ####,please provide '^####'

Defines if the pattern set under pattern should be negated or not. Default is false.

multiline.negate: true

Match can be set to "after" or "before". It is used to define if lines should be append to a pattern

that was (not) matched before or after or as long as a pattern is not matched based on negate.

Note: After is the equivalent to previous and before is the equivalent to to next in Logstash

multiline.match: after

[wwalker (Walker)]

I don't believe you need the parenthesis around each of the values. Have you tried just pattern => "SEVERE|ERROR|WARN|INFO|DEBUG" ?

[cawoodm (Marc)]

Yes, that seems to help thanks!