Yet another funky grokparsefailure

Hi gang,

Completely out of my wits on this one...

Here's a log line (NGINX custom format):

13.88.158.6 - - [14/Mar/2018:00:02:03 +0000] "GET /issue/wsfed?wa=wsignout1.0 HTTP/1.0" 499 0 "https://msft2017r2-wus2-prd.whatever.com/app.aspx" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36 Edge/16.16299" "-" "pass.whatever.com" sn="passproxy-prod.whatever.com" rt=49.171 ua="13.88.158.6:443" us="-" ut="-" ul="0" cs=-Netherlands NL NH

It can be parsed this way with grok:

filter {
grok {
match => [ "message","%{COMBINEDAPACHELOG}+%{GREEDYDATA:extra_fields}"]
overwrite => [ "message" ]
}

but the result is not accurate.

Here's an expression that will fly in https://grokdebug.herokuapp.com and other grok debuggers:

%{IPORHOST:remote_addr} %{USER:ident} %{USER:auth} [%{HTTPDATE:timestamp}] \"%{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion}\" %{INT:status} %{NUMBER:bytes_sent} \"%{DATA:http_referer}" \"%{DATA:http_user_agent}" \"%{DATA:http_x_forwarded_for}\" \"%{DATA:host}\" sn=\"%{DATA:server_name}\" rt=%{DATA:request_time} ua=\"%{DATA:upstream_addr}\" us=\"%{DATA:upstream_status}\" ut=\"%{DATA:upstream_response_time}\" ul=\"%{DATA:upstream_response_length}\" %{GREEDYDATA:extra_fields}

... but that gets me a _grokparsefailure in logstash.

Any idea how I shall build the "match" statement in logstash.conf so that my (legal) expression does not result in a _grokparsefailure ?

I've tried everything, working my way up from what is working. Every time I try to add something here I get this dreaded grokparsefailure:

match => [ "message","%{COMBINEDAPACHELOG}[ADD-SOMETHING-HERE]+%{GREEDYDATA:extra_fields}"]

Thanks much,

Fred

You need to escape brackets in the Logstash config.
eg.
[%{HTTPDATE:timestamp}] --> \[%{HTTPDATE:timestamp}\]

Thanks atira !

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.