Grok multiline parse failures

[Jordan160]

Hello,

I am fairly new to ELK stack and currently am having issues with GROK filter.

My pattern matches when using http://grokconstructor.appspot.com, but it still seems to be breaking on a particular log pattern that has multiple lines/spaces.

Example log:

2017-10-19 13:35:03,732 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-7) MSC000001: Failed to start service jboss.undertow.listener.default: org.jboss.msc.service.StartException in service jboss.undertow.listener.default: Could not start http listener
at org.wildfly.extension.undertow.ListenerService.start(ListenerService.java:150)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1948)
at org.jboss.msc.service.ServiceControllerImpl$StartTask.run(ServiceControllerImpl.java:1881)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

Pattern I'm using:

match => [ message => "(?m)%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:loglevel}%{SPACE}[%{DATA:class}]\s(%{DATA:thread})\s(?(.|\r|\n)*)" ]

Any suggestions is greatly appreciated.

[magnusbaeck (Magnus Bäck)]

Why do you need all the (?(.|\r|\n)*) stuff at the end? Don't you want to catch everything after the thread name in one field?

Don't use DATA in multiple places. You don't need it. Use (?<class>[^\]]+) and (?<thread>[^)]+) instead. (Side node: I suspect "class" is the wrong fieldname. I suspect it's actually the logger name, although the logger name usually happens to be the same as the class name.)

[Jordan160]

@Jordan160

(?(.|\r|\n)*)" ]

Thanks for the reply.

I've replaced class/thread with your suggestions. My goal is to capture everything after thread into a field "logmessage".

I've tried: (?(.|\r|\n)*) and %{GREEDYDATA:logmessage} but both appear to not parse the next lines and give me the grokparseerror:

[magnusbaeck (Magnus Bäck)]

I'm pretty sure GREEDYDATA matches newlines. What if you leave out that part at the end, does that make the grok expression work?

[Jordan160]

It does not work because it tries to evaluate the next line as a new line still:

I've tried switching the pattern between pattern => "^[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}" and pattern => "^(?!201)" but it is still not parsing these lines correctly.

[magnusbaeck (Magnus Bäck)]

Oh, so it's really a multiline codec problem. That wasn't clear at all. Don't use the multiline codec with the Beats input. Do multiline processing on the Filebeat side. Its documentation contains an example of almost exactly your kind of log.

[Jordan160]

That did the trick, thanks a bunch for your help!