GREEDYDATA and newlines

Elastic Stack Logstash
1

I have a log file from a java program coming from filebeat. Some of the events have stacktraces and so are multiline. I'm using the multiline option in filebeat and a grok filter in logstash to parse the event. Everything works well when I end the pattern in %{GREEDYDATA:logmessage} however I'd like to split the "logmessage" at the first newline character and keep the remainder as "stacktrace". What's confusing me is that the "logmessage" field already contains the full stacktrace including "\n" characters! This post, http://stackoverflow.com/questions/26474873/how-do-i-match-a-newline-in-grok-logstash, states "All GREEDYDATA is is .*, but . doesn't match newline". How does my logmessage field contain newline characters?

I'd like to do something like this:

SNIP%{GREEDYDATA:logmessage}\n%{GREEDYDATA:stacktrace}

2

Further, how did my log file with multiple lines end up being a single line with literal \n characters? Is that the work of the multiline option in filebeat?

3

Further, how did my log file with multiple lines end up being a single line with literal \n characters? Is that the work of the multiline option in filebeat?

Yes, that's why you'd use the multiline option.

4

Ok, thanks Magnus. How then do I split into two fields something like this:

logstash \nrules!

That is, how do I split on the \n?

5

On a related topic, I'm doing this because sometimes "logmessage" includes a huge stacktrace and for whatever reason, it displays blank in kibana. I guess there's a maxlength for a field or something? My thought was to split the logmessage from the stacktrace.

6

That is, how do I split on the \n?

The mutate filter's split option can certainly split a string, but then you'll split on all newline characters and you just want to split on the first one. It could be that the best way is to write a small Ruby snippet in a ruby filter.

7

Thanks @magnusbaeck. Any thoughts on why large stacktraces appear blank in Kibana? Is there a maximum field size?