Grok parse error

Elastic Stack Logstash
1

So I'm getting a parse error on my grok filter.

I have gone to the following url to debug it: http://grokdebug.herokuapp.com/

My input is this:

66.249.69.48|-|2015-11-05T10:57:51-06:00|/northwest-a-tacs-camo-seat-covers/?utm_campaign=product_ads&utm_source=google&utm_medium=cpc&utm_content=854976&productid=854976&cparam=2346273|499|0|http://www.realtruck.com/northwest-a-tacs-camo-seat-covers/?utm_campaign=product_ads&utm_source=google&utm_medium=cpc&utm_content=854976&productid=854976&cparam=2346273|Mozilla/5.0 (iPhone; CPU iPhone OS 8_3 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12F70 Safari/600.1.4 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)|0.237|-|.

My grok filter is this:

%{IP:visitor_ip}\|[^|]+\|%{TIMESTAMP_ISO8601:entryDateTime}\|%{URIPATH:url}%{URIPARAM:query_string}?\|%{INT:http_response}\|%{INT:response_length}\|(?<http_referrer>[^|]+)\|(?<user_agent>[^|]+)\|%{BASE16FLOAT:request_time}\|%{BASE16FLOAT:upstream_response_time}

I am expecting a time stamp to come out of this, but it is getting a "No Matches" and causing logstash grok parse errors.

I am at a loss as it works part of the time. Anyone have a quick second to look at this?

This is a snapshot of the error in Kibana:

groc.PNG1569×468 10.9 KB

This is a snapshot of one that works:

working.png1570×766 45.5 KB

2

The final %{BASE16FLOAT:upstream_response_time} doesn't match the hyphen that it's getting. Replace with (%{BASE16FLOAT:upstream_response_time}|-).

1 Like
3

Magnus beat me to it. However to add the reason it works sometimes and not others means somtimes you are getting a value back for the upstream response time. Otherwise you would just get a null for that value.

4

Hey Guys,

@Magnus, thanks for that input!!

When looking through nginx HTTP codes, 499 is saying that google closed the connection before we could get it all setup. So I think a null value or 'zero' would be just fine because we wouldn't get any statistics on it anyway.

Would it be smarter to put a regex test in there to say if the value that is returned is "-", input 0, and then if it isn't put in the value?