Logstash grokparsefailure

[sandikata (Росен Александров)]

Hello!
Is it there any way (human readable) to debug what is causing grokparsefailure?
There's my nginx filter conf

filter { if "nginx_access" in [tags] {
  grok {
    patterns_dir => ["/etc/logstash/patterns"]
    match => { "message" => "%{NGINXACCESS}" }
	}

#grok { match => { "request" => "%{NOTSPACE:request_file}\?" } }
grok { match => { "request" => "%{URIPATH:request_file}%{URIPARAM:request_params}" } }

mutate { 
	remove_field => [ "bytes", "httpversion", "agent.ephemeral_id", "agent.id", "agent.name", "agent.hostname", "agent.type", "agent.version" ] 
	}
}

	 if ![x_first_ip] {
   geoip {
    source => "clientip"
       target => "geoip"
    database => "/usr/share/GeoIP/GeoIP2-City.mmdb"
    fields => ["continent_code", "country_name", "country_code3", "region_name", "city_name", "postal_code", "region_code", "location"]
   }
   ip2proxy {
    source => "clientip"
	database => "/usr/share/IP2Proxy/IP2PROXY-IP-PROXYTYPE-COUNTRY.BIN"
   }
  } else {
   ip2proxy {
    source => "x_first_ip"
	database => "/usr/share/IP2Proxy/IP2PROXY-IP-PROXYTYPE-COUNTRY.BIN"
   }
   geoip {
    source => "x_first_ip"
    database => "/usr/share/GeoIP/GeoIP2-City.mmdb"
    fields => ["continent_code", "country_name", "country_code3", "region_name", "city_name", "postal_code", "region_code", "location"]
   }
  }
#}
#  useragent {
#  source => "user_agent"
#	}
}

filter { if "nginx_errors" in [tags] {
  grok {
	add_tag => ["nginx_errors"]
}
}
}

And the pattern which i use.

NGINXACCESS %{IPORHOST:http_host} %{IPORHOST:clientip} \[%{HTTPDATE:timestamp}\] \"(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\" %{NUMBER:response_code} (?:%{NUMBER:bytes}|-) \"%{DATA:referrer}\" \"%{DATA:useragent}\" \"%{DATA:request_body}\" \"(-|(?<x_forwarded_for>%{IP:x_first_ip}(?:, [^\s,]+)*)?)\" \"%{DATA:x_server}\" \"(-|%{DATA:api_key})\" %{NUMBER:request_time}

I am trying add some more thinks, and cannot find which is the problem.

[sandikata (Росен Александров)]

Also another think.
Is it there any difference between built-in grok debugger and https://grokdebug.herokuapp.com/
Because some logs are not parsed properly, if i check them on debugger from the link everything is parsed properly, but in kibana built-in debugger just says grokparsefailure

First example have been taken from built-in

Second example have been taken from the link.

What's could be the difference, because in kibana it won't parse anyway.

[Badger]

@sandikata

Is it there any way (human readable) to debug what is causing grokparsefailure?

My suggestion for building complex grok patterns is here.

Note that grok debuggers (including kibana) and grok itself sometimes interpret ambiguous patterns differently (and almost every pattern that uses DATA, or especially GREEDYDATA, is ambiguous).

[sandikata (Росен Александров)]

Thank You for tips.