Logstash date timezone

Elastic Stack Logstash
1

I want to parse date-time records with logstash date filter. My records have Asia/Tehran time zone.
After the recent changes in the time zone in Iran and the removal of DST from it, apparently my date of records does not parse correctly.
For example, June 10, 2021 at 12:00 Tehran time should be 7:30 UTC, and June 10, 2023 at 12:00 Tehran time should be 8:30 UTC, but both are 7:30 UTC.

My logstash version is 8.8.1 and I tested it with versions 8.8.0 and 7.14.0 and the result was the same.

sample code:

echo "10 June 2021 12:00:00.000" | bin/logstash -e "filter { date { match => ['message', 'dd MMMM YYYY HH:mm:ss.SSS'] target => timeid timezone => 'Asia/Tehran' } }"
result:
"timeid" => 2021-06-10T07:30:00.000Z

echo "10 June 2023 12:00:00.000" | bin/logstash -e "filter { date { match => ['message', 'dd MMMM YYYY HH:mm:ss.SSS'] target => timeid timezone => 'Asia/Tehran' } }"
result:
"timeid" => 2023-06-10T07:30:00.000Z
2

So the country changed own time zone?
Well someone in Ruby/Java should correct that in the code.
The documentation: The valid IDs are listed on the Joda.org available time zones page.

Until then you can use hardcoded values like timezone => "+03:30" and add few IFs which will handle that.

The code with timezone => "+03:30" means UTC+03:30(pure sum), it doesn't matter is daylight saving time or not. With the JODA timezone, DST is corrected automatically.
If I remember correctly, I had tested for Turkey is corrected, it's always UTC+03:00 now (from 2017), but i.e. Berlin has UTC+01:00 or UTC+02:00 in DST.

3

Yes, Iran abolished its DST(Daylight saving time) from March 2023.

But the main problem is that the time of my records are from 1980 to 2023 and I want the date filter to behave correctly with all of them. So I can't use timezone => "+03:30" on all records.

4

The tz data should know the historical record of timezone changes. For example, zdump -i Australia/Queensland show that DST was used in Queensland during WW1 and WW2 (very common) and tried in the 1990's (it was then rejected by referendum)

zdump -i Australia/Queensland

TZ="Australia/Queensland"
-       -       +101208 LMT
1894-12-31      23:47:52        +10     AEST
1917-01-01      03      +11     AEDT    1
1917-03-25      02      +10     AEST
1942-01-01      03      +11     AEDT    1
1942-03-29      02      +10     AEST
1942-09-27      03      +11     AEDT    1
1943-03-28      02      +10     AEST
1943-10-03      03      +11     AEDT    1
1944-03-26      02      +10     AEST
1971-10-31      03      +11     AEDT    1
1972-02-27      02      +10     AEST
1989-10-29      03      +11     AEDT    1
1990-03-04      02      +10     AEST
1990-10-28      03      +11     AEDT    1
1991-03-03      02      +10     AEST
1991-10-27      03      +11     AEDT    1
1992-03-01      02      +10     AEST

The same for Asia/Tehran (or IRST or IRDT) does not have history

zdump -i TZ=Asia/Tehran

TZ="TZ=Asia/Tehran"
-       -       +00     ""

The release notes for the 2022b tzinfo (August 2022) states that "Iran plans to stop observing DST permanently, after it falls back on 2022-09-21" but the tz files in my system do not reflect that (or changes from 1935 onwards).

I do not know why the tzinfo file for Tehran do not have the history.

5

I think it needs to be zdump -i Asia/Tehran, it works on my system and the last change is on 2022-09-21 reverting back to +0330

2020-03-21	01	+0430		1
2020-09-20	23	+0330
2021-03-22	01	+0430		1
2021-09-21	23	+0330
2022-03-22	01	+0430		1
2022-09-21	23	+0330

Are you using the bundled JDK or are you using another JDK? If you are using another JDK maybe the version you are using does not have the timezone update.

1 Like
6

I am using bundled jdk

7

Not sure what could be the issue, the bundled JDK for version 8.8.1 uses Temurin-17.0.7+7, which has tzdata version 2022g

$ strings /opt/logstash-8.8.1/jdk/lib/tzdb.dat | head -n2
TZDB
2022g

From the tzdata changelog the Iran DST change was applied on 2022b.

I think you should open an issue on Logstash Github so Elastic can check what may be happening.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.