Timezone for date filter seems not working


(Z Afshar) #1

Hello,
I am newbie to logstash and have a problem with date filter.
I have a historical csv file with information like this in each line:

2015-06-10 16:00:00.017,10.0.0.100,192.168.1.1

the time specified above is in my local timezone (Asia/Tehran - UTC+3:30 )
I made a configuration file for testing:
input {
stdin{}
}
filter {
grok {
match => {
"message" => "%{TIMESTAMP_ISO8601:logTime},%{IP:client},%{IP:server}"
}
}
date {
timezone => "Asia/Tehran"
match => [ "logTime", "YYYY-MM-dd HH:mm:ss.SSS"]
}
}
output {
stdout {
codec => rubydebug
}
}
when I send the log above to logstash through stdin the logstash does not change the timestamp and show the UTC time:
2015-06-10 16:00:00.017,10.0.0.100,192.168.1.1
{
"message" => "2015-06-10 16:00:00.017,10.0.0.100,192.168.1.1",
"@version" => "1",
"@timestamp" => "2015-06-10T11:30:00.017Z",
"host" => "lab-pc",
"logTime" => "2015-06-10 16:00:00.017",
"client" => "10.0.0.100",
"server" => "192.168.1.1"
}

removing the timezone from date filter returns same result.
any help would be appreciated.


#2

Hello,
Your configuration and result are correct but it seems you are confused by the behaviour of the date filter.
Sorry that the documentation does not make it easy to understand.
In logstash @timestamp is always stored in UTC, so the date filter will always convert the value to UTC.
Specifying the timezone in the configuration allow you to make the correct conversion for datetime string that do not contains the timezone information.
In your case as Iran is now following summer time => UTC + 4:30 it will get converted to UTC 2015-06-10T11:30:00.017Z as you observed.

Is it more clear now ?


(system) #3