Grok parse rule succeeding in debugger, but not in Logstash

[Nik_Gupta]

Hey, would really appreciate some help debugging this.

Example log line:
2019-02-24 17:58:30,396 INFO [i.b.logging.LoggingComponent] [] [] [] : The log message

Parse rule that succeeds in the Grok debugger but fails for all logs in logstash:
%{DATE:date} %{TIME:timestamp} %{LOGLEVEL:level} \[(%{DATA:logger})?\] %{GREEDYDATA:content}

Parse rule that succeeds on for all logs on both:
%{DATE:date} %{TIME:timestamp} %{LOGLEVEL:level} %{GREEDYDATA:content}

Logstash is being fed by filebeat with a multiline configuration to handle stacktraces.

Thanks in advance,
Nik

[pr0b3l (Andy)]

Hello Nik.

Why you use [(%{DATA:logger})?]?

Try to use without ?

[Nik_Gupta]

Hey Andy,

It's optional whether or not the logger name is inside the square brackets, hence the question mark.

However, even if I remove this, parsing still fails for all lines - i.e. including ones where the logger name is in the square brackets.

Cheers,
Nik

[Badger]

You could use dissect to replace grok

dissect { mapping => { "message" => "%{ts} %{+ts} %{loglevel} [%{logger}] %{restOfLine}" } }

[Nik_Gupta]

Hey Badger,

Thanks - I actually ended up doing exactly that already (should have said).
Will set-up a minimal example and raise an issue on GH.

Cheers,
Nik