Having trouble with the date filter for iis logs

[Doron_Raz]

Continuing the discussion from Date Filter Not working parsing IIS logs:

@Peter_Dwyer

I'm having trouble with the date filter. it doesn't seem to be updating the timestamp correctly.

filter {
  if [type] == "iis" {
    grok {
      match => {"message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:serviceName} %{IP:serverIP} %{WORD:method} %{URIPATH:uriStem} %{NOTSPACE:uriQuery} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientIP} %{NOTSPACE:protocolVersion} %{NOTSPACE:userAgent} %{NOTSPACE:cookie} %{NOTSPACE:referer} %{NOTSPACE:requestHost} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"}
    }
    date {
        match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss", "ISO8601" ]
        target => "@timestamp"
        remove_field => ["log_timestamp"]
    }

  }
}

I get the correct timestamp in the debugger, but not in Kibana.
here is an example of the parsed log in the debugger:

{
      "uriQuery" => "-",
       "message" => "2018-06-02 23:50:10 10.1.0.250 GET /Images/Homepage/Promotion/201da2a5.jpg - 443 52332470000003 10.1.10.85 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 200 0 0 31\r",
        "method" => "GET",
     "timetaken" => 31,
      "username" => "52332470000003",
      "serverIP" => "10.1.0.250",
    "@timestamp" => 2018-06-02T23:50:10.000Z,
          "port" => "443",
          "host" => "MyELK",
      "response" => "200",
   "subresponse" => "0",
          "path" => "/home/doronr/data/u_ex180602_03.log",
"clientHostname" => "10.1.10.85",
       "uriStem" => "/Images/Homepage/Promotion/201da2a5.jpg",
          "type" => "iis-w3c",
          "tags" => [
    [0] "_grokparsefailure"
],
 "win32response" => "0",
     "userAgent" => "Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36",
      "@version" => "1",
      "clientIP" => "10.1.10.85"

}

[Doron_Raz]

and here is the iis log entry:

2018-06-02 23:50:10 10.1.0.250 GET /Images/Homepage/Promotion/201da2a5.jpg - 443 52332470000003 10.1.10.85 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 200 0 0 31

[Badger]

@Doron_Raz

I get the correct timestamp in the debugger, but not in Kibana.

What timestamp do you get in Kibana for that message?

[Doron_Raz]

Thank you for asking...
Now I see that Kibana timestamp is 2 hours later than the log.
So I added the correct timezone to the date filter and it works.

Thanks