Date showing as UTC and indexed like it and not in local time

Elastic Stack Kibana
1

Hello,

I have some auth0 logs streamed via Logstash where I use date filter to capture the date from the message itself.
In the message the date is in the form of: 2018-02-19T00:35:32.145Z

My date filter looks like this:
date {
match => [ "date", "YYYY-MM-dd'T'HH:mm:ss.SSS'Z'"]
timezone => "Australia/Melbourne"
}

In Kibana: Advanced settings set to: dateFormat:tz - Browser (I was playing and changing it to Australia/Melbourne but no luck)

Indexed dates are:
date coming from the message: 2018-02-19T00:35:32.145Z
date created by the "date" plugin: 2018-02-18T13:35:32.145Z

current AEDT date: February 19th 2018, 11:35:32

In Kibana I search for events and search is going by the current Australian time (That's also what I want it to be). But events appearing in the past due to the UTC time in the index.

How can I fix it - The search for events and the @timestamp presentation in local time?

2

2018-02-19T00:35:32.145Z is the same as 2018-02-18T13:35:32.145Z. The former is just +11 which is what TZ Australia/Melbourne is.

3

Yeah but in Kibana it doesn't reflect the time in local time +11 hours as you've said.
So events appear to be in the past.
Any idea on how to fix it?

4

Kibana will use local system time from the browser and then convert things accordingly.
It'd be worth checking that.

5

Yeah, as I've mentioned in my original post I've tried to:
In Kibana: Advanced settings set to: dateFormat:tz - Browser (I was playing and changing it to Australia/Melbourne but no luck)

But nevertheless events appear in the past. so I have to use "target" to move message timestamp field to "message_time":
date {
match => [ "date", "YYYY-MM-dd'T'HH:mm:ss.SSS'Z'"]
timezone => "Australia/Melbourne"
target => "message_date"
}

See for yourself:

message {"date":"2018-02-20T01:35:46.415Z","type":"sepft","description":"","connection":"MyAccountUsers","connection_id":"con_lTBdDeovGBu44Bk1","client_id":"qRQRF4upIkHN5tJK72nukXJQKGidvmgk","client_name":"LegacyMobileApp","ip":"13.55.240.10","user_agent":"Apache-HttpClient 4.5.2 / Other 0.0.0","user_id":"auth0|MyAccountUser|U00000092030","user_name":"bro59@gmail.com","audience":"https://digitalapi","scope":"openid email","_id":"90020180220013546415749875000555906985398907141626003458","log_id":"90020180220013546415749875000555906985398907141626003458","isMobile":false}

message_date February 20th 2018, 01:35:46.415
post_date 1,519,090,828,356
@timestamp February 20th 2018, 12:40:28.579 <- THIS IS INGESTION TIME (I WOULD LIKE THIS TO BE message_date but not in the past)

Here is what Logstash is passing to ElasticSearch:
"message_date" => 2018-02-19T14:35:46.415Z,
"post_date" => 1519090828356,
"@timestamp" => 2018-02-20T01:40:28.579Z,

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.