Logstash Grok pattern GREEDYDATA does not match


(Nick) #1

Hello there

I'm currently building a pipeline with different grok patterns. For the most part it's properly working already but I'm having issues with some Jira logs:

Log Message:
2018-11-21 12:50:23,950 http-nio-8080-exec-4 INFO anonymous 769x323x1 1hhvqv1 10.195.161.19 /secure/SetupLicense.jspa [c.a.j.p.devstatus.upgrade.UpgradeTask_Build02] Running Upgrade task to add searcher for development field.

Grok Pattern:
(?%{YEAR}-%{MONTHNUM}-%{MONTHDAY}%{SPACE}%{TIME}) %{JIRA_THREAD:thread} %{WORD:loglevel} %{WORD:user} %{JIRA_HTTP_ID:http_request_id} %{JIRA_HTTP_ID:http_session_id} %{IP:ip} %{UNIXPATH:path} %{JIRA_CLASS:class} %{GREEDYDATA:message}

Custom Patterns:
JIRA_THREAD [A-Za-z0-9-/:]+
JIRA_HTTP_ID [A-Za-z0-9-]+
JIRA_CLASS [A-Za-z0-9.]+

The GREEDYDATA:message part doesn't match and I don't understand why. I've checked it with the inbuilt Kibana grok debugger. Can someone please explain this behavior to me?

Thanks in advance!


(Lewis Barclay) #2

What does the result look like?


(Nick) #3

The grok debugger output? It's empty and the error is: "Provided Grok patterns do not match data in the input"


(Lewis Barclay) #4

Try this:

%{TIMESTAMP_ISO8601:date} %{JIRA_THREAD:thread} %{WORD:loglevel} %{WORD:user} %{JIRA_HTTP_ID:http_request_id} %{JIRA_HTTP_ID:http_session_id} %{IP:ip} %{URIPATH:path} %{JIRA_CLASS:class} %{GREEDYDATA:message}

Custom pattern:

JIRA_THREAD %{WORD}-%{WORD}-%{INT}-%{WORD}-%{INT}
JIRA_HTTP_ID [A-Za-z0-9-]+
JIRA_CLASS [A-Za-z0-9.\[\]_]+

(Nick) #5

Oh, I missed the underscore in the JIRA_CLASS pattern. Thank you for the fast help!