Doron_Raz
(Doron Raz)
June 10, 2018, 1:54pm
1
Continuing the discussion from Date Filter Not working parsing IIS logs :
I'm having trouble with the date filter. it doesn't seem to be updating the timestamp correctly.
filter {
if [type] == "iis" {
grok {
match => {"message" => "%{TIMESTAMP_ISO8601:log_timestamp} %{WORD:serviceName} %{IP:serverIP} %{WORD:method} %{URIPATH:uriStem} %{NOTSPACE:uriQuery} %{NUMBER:port} %{NOTSPACE:username} %{IPORHOST:clientIP} %{NOTSPACE:protocolVersion} %{NOTSPACE:userAgent} %{NOTSPACE:cookie} %{NOTSPACE:referer} %{NOTSPACE:requestHost} %{NUMBER:response} %{NUMBER:subresponse} %{NUMBER:win32response} %{NUMBER:bytesSent} %{NUMBER:bytesReceived} %{NUMBER:timetaken}"}
}
date {
match => [ "log_timestamp", "YYYY-MM-dd HH:mm:ss", "ISO8601" ]
target => "@timestamp"
remove_field => ["log_timestamp"]
}
}
}
I get the correct timestamp in the debugger, but not in Kibana.
here is an example of the parsed log in the debugger:
{
"uriQuery" => "-",
"message" => "2018-06-02 23:50:10 10.1.0.250 GET /Images/Homepage/Promotion/201da2a5.jpg - 443 52332470000003 10.1.10.85 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 200 0 0 31\r",
"method" => "GET",
"timetaken" => 31,
"username" => "52332470000003",
"serverIP" => "10.1.0.250",
"@timestamp" => 2018-06-02T23:50:10.000Z,
"port" => "443",
"host" => "MyELK",
"response" => "200",
"subresponse" => "0",
"path" => "/home/doronr/data/u_ex180602_03.log",
"clientHostname" => "10.1.10.85",
"uriStem" => "/Images/Homepage/Promotion/201da2a5.jpg",
"type" => "iis-w3c",
"tags" => [
[0] "_grokparsefailure"
],
"win32response" => "0",
"userAgent" => "Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36",
"@version" => "1",
"clientIP" => "10.1.10.85"
}
Doron_Raz
(Doron Raz)
June 10, 2018, 1:56pm
2
and here is the iis log entry:
2018-06-02 23:50:10 10.1.0.250 GET /Images/Homepage/Promotion/201da2a5.jpg - 443 52332470000003 10.1.10.85 Mozilla/5.0+(Windows+NT+10.0;+Win64;+x64)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/66.0.3359.181+Safari/537.36 200 0 0 31
Badger
June 10, 2018, 4:52pm
3
What timestamp do you get in Kibana for that message?
Doron_Raz
(Doron Raz)
June 11, 2018, 6:49am
4
Thank you for asking...
Now I see that Kibana timestamp is 2 hours later than the log.
So I added the correct timezone to the date filter and it works.
Thanks
system
(system)
Closed
July 9, 2018, 6:50am
5
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.