_grokparsefailure: All fields are here but still get the error

[faulander]

Here is the GROK:
%{IPORHOST:remote_ip} %{USER:ident} %{USER:auth} \[%{HTTPDATE:access_time}\] TLSv%{DATA:tls} %{NOTSPACE:RSA} \"%{WORD:method} %{NOTSPACE:url} %{DATA:protocol}\/%{NUMBER:version}\" %{NUMBER:Statuscode} (?<size>\d+|\-) %{NUMBER:time}

Here is the test line:
10.99.99.99 - - [10/Oct/2018:12:25:37 +0200] TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 "GET /xxxxxxxxxx/pub/images/header_logo.png HTTP/1.1" 200 6349 31

Works perfectly in Grok Debugger. Output of Logstash:
{
"access_time" => "10/Oct/2018:12:25:37 +0200",
"message" => " 10.99.99.99 - - [10/Oct/2018:12:25:37 +0200] TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 "GET / HTTP/1.1" 302 225 0",
"host" => "xxx.xxx.local",
"tags" => [
[0] "_grokparsefailure"
],
"method" => "GET",
"time" => "0",
"auth" => "-",
"@version" => "1",
"remote_ip" => "10.99.99.99",
"Statuscode" => "302",
"version" => "1.1",
"size" => "225",
"url" => "/xxxxxxxxxx/pub/images/header_logo.png",
"RSA" => "ECDHE-RSA-AES256-GCM-SHA384",
"protocol" => "HTTP",
"@timestamp" => 2018-10-12T09:45:01.181Z,
"ident" => "-",
"tls" => "1.2"
}

[AquaX (Andreas Helmer)]

Looks like all of your fields are being populated... do you have another grok filter in your config?
You could always go one field at a time and just put %{GREEDYDATA} afterwards, adding a more specific filter each time until you find your parsing error.

ie..
%{IPORHOST:remote_ip}%{GREEDYDATA:TheRest}
%{IPORHOST:remote_ip} %{USER:ident} %{GREEDYDATA:TheRest}
%{IPORHOST:remote_ip} %{USER:ident} %{USER:auth}%{GREEDYDATA:TheRest}
etc...

Takes a bit of debugging and time, but this strategy has worked for me, especially when the Grok Debugger site has checked out ok.

[faulander]

yes, i will need to try that ...

Thanks,

Harald

[Makra (Makara)]

The grok works perfectly in my system without any grokparsefailure.

{
            "RSA" => "ECDHE-RSA-AES256-GCM-SHA384",
         "method" => "GET",
           "auth" => "-",
          "ident" => "-",
        "message" => "10.99.99.99 - - [10/Oct/2018:12:25:37 +0200] TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 \"GET /xxxxxxxxxx/pub/images/header_logo.png HTTP/1.1\" 200 6349 31",
        "version" => "1.1",
            "url" => "/xxxxxxxxxx/pub/images/header_logo.png",
           "path" => "/home/data/sample.log",
       "protocol" => "HTTP",
     "@timestamp" => 2018-10-13T04:57:17.498Z,
      "remote_ip" => "10.99.99.99",
           "size" => "6349",
       "@version" => "1",
           "host" => "localhost",
            "tls" => "1.2",
           "time" => "31",
     "Statuscode" => "200",
    "access_time" => "10/Oct/2018:12:25:37 +0200"
}