Change time zone using date filter

Elastic Stack Logstash
1

Hi there,

i have a problem with timezone in date filter. so this is the situation:
i have a field contain an epoch timestamp like this

gambar
gambar1103×253 17.9 KB

i try to convert it using date filter like this but it didn't work

mutate{
        convert => {
        "[service][updated_at]" => "string"
        }
        strip => ["[service][updated_at]"]
}
   date {
        match => ["[service][updated_at]", "UNIX_MS"]
        target => "[service_updated_at]"
        timezone => "+07:00"
   }

i already define timezone there but i don't know why the result becomes like this:

gambar
gambar1112×274 20.1 KB

for comparison, if i try to paste the number in epoch site, it tells me the correct date
gambar

anybody knows how to fix this? please help.

Thanks

2

This is not correct and it is a common confusion, the timezone option in the date plugin on logstash should not be used to change the timezone of the value, it should be used when you have a date string without any timezone information and the date string is not on UTC because both Logstash and Elasticsearch works with UTC.

If your date string already has timezone information you should not use the timezone option of the date filter.

Since your date string is on unix epoch, it is already on UTC, so you cannot use the timezone option in the logstash date filter, using it will add a 7 hours offset to the UTC date.

So the date 2023-02-09 10:41:21 UTC will become 2023-02-09 17:41:21 UTC, and when Kibana shows up dates it will per default convert the UTC value to the browser timezone, if you are on a +0700 timezone, Kibana will add this to the UTC date you will have 2023-02-10 00:41:21 on Discover.

You need to remove the timezone option from the date filte as your date string is already in UTC.

1 Like
3

sorry, maybe i forgot to explain, but that's my goal to change the timezone to GMT +7. that's why I added the timezone option and thanks for your response. i have solved this problem. I tried changing UNIX_MS to UNIX and it worked for me. I don't know why this happened? I see from the documentation there is no significant difference between UNIX and UNIX_MS. Why do you think this works by matching the data to UNIX?

4

In general I would recommend against this. elasticsearch expects all times to be in UTC (if you were feeding some other downstream system then I would have no issue at all with doing this). Programs consuming data from elasticsearch will expect times to be in UTC (kibana will change them to the browser's timeone by default). If you are sure everyone consuming the data (now or in the future) is happy with the timezone shift then you should be OK, but, as I said, I recommend against it.

One is in seconds, one in milliseconds, at the time of writing UNIX would be 1676431668, UNIX_MS would be 1676431668000.

5

ok, i think it's pretty clear. thanks for the explanation @Badger @leandrojmp

6

I also would not recommend that, you are adding a timezone offset to a date that has no timezone offset.

Elasticsearch works with UTC dates and unix epoch is already in UTC, when you add a timezone to a UTC date you are changing the original date saying that it happened on a different time and this can leads to confusion or even auditing problems if you have any audit on your systems.

But if you are sure that everyone that uses your systems knows about this change, then it would not be so much of a problem.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.