Kibana displays wrong time

[vikramaddagulla]

Hello,

I have a log file and I am sending that from beats --> logstash --> elastic --> kibana.

Below are two lines in the log file :

172.27.88.123 2020-04-15T02:54:15.054Z GET /em 302 311 0.002
172.27.88.123 2020-04-15T02:54:16.054Z GET /em/console/home 302 533 0.004

Grok is as below :

match => ["message", "%{IPORHOST:clientip}%{SPACE}%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{WORD:method}%{SPACE}%{NOTSPACE:request}%{SPACE}%{NUMBER:response}%{SPACE}%{NUMBER:bytes}%{SPACE}%{NUMBER:timetaken}"]

And then the below lines to convert the timestamp into @timestamp

mutate {
add_field => { "newtimestamp" => "%{timestamp}" }
remove_field => ["timestamp"]
}
date {
#match => [ "newtimestamp" , "yyyy-MM-dd'T'HH:mm:ss'Z'" ]
match => [ "newtimestamp" , "ISO8601" ]
timezone => "America/New_York"
target => "@timestamp"
}

The timestamp value in the index looks like below :
"@timestamp" : "2020-04-15T02:39:38.039Z",

However, when I try to view this in Kibana, The time in the indexed data goes back by 4 hours.

I tried to change the settings of kibana to America/New_York but no luck.

Any suggestions ??

[Badger]

elasticsearch always stores timestamps in UTC. If a timestamp explicitly includes a timezone (as yours does, since it has a trailing Z) then the date filter ignores the timezone option.

By default kibana will display the timestamp in the timezone of the browser. You can change that.

[elasticforme (Sachin)]

I have learn hard way not to change that @timestamp. keep it as is and let kibana handle it stuff.

assigning proper Timezone is only useful if your data is being pulled via sql because it does not converts your time back to your localtime. it will print as is.

[vikramaddagulla]

Thank you..

I was able to fix the issue by removing the Z in the timestamp.

The timestamp that was being passed was in EST and due to the presence of Z, it was always considered as UTC and hence the wrong time was getting processed.