Parsing logs with stack trace


(maverick) #1

Hi,

I have a situation where I am parsing logs that usually are in a structured format and am able to parse it and index them. But when there is an error, there is stacktrace which spans multiple lines and hence the grok pattern fails. An example is as follows:

Grok Pattern:
match => [
"message" , "<%{NUMBER:syslog_pri}>%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program} %{TIMESTAMP_ISO8601:timestamp} [%{DATA:loglevel}] %{DATA:class} [%{DATA:operation}] (user=%{DATA:userid}) (?m)%{GREEDYDATA:stacktrace}"

Log message example:
2018-11-12 12:15:11,893 [ INFO] Dummy Message: [Type] (user=abc, key=xyz) SKIP

So the above message gets parsed without any issues. But the following it doesnt parse the whole stacktrace:

##################################################################
2018-11-12 12:15:13,688 [ ERROR] Dummy Message [Error] (user=abc) Unhandled exception encountered...
Bla bla bla
bla Bla bLna
bla bla bla
#################################################################

Any way to include all the stacktrace?