Logstash date not matching

Hey all,
I'm trying to work out why I get a grok parse failure on date here when using these two as input

2014-10-05 02:59:25 Works
2015-10-05 02:59:25 Does not work

timestamp.conf

   input {
        stdin { }
    }


    filter {
     grok {
        match => { "message" => "%{TIMESTAMP_ISO8601:logdate}" }
     }}

filter {
	date {
	match => [ "logdate", "yyyy-MM-dd HH:mm:ss" ]
	target => "@timestamp"
}}
output {
    stdout { codec => rubydebug }
}

./logstash -f timestamp.conf -e

Not Working

2014-10-05 02:59:25
{
      "@version" => "1",
          "host" => "elastic18",
       "logdate" => "2014-10-05 02:59:25",
          "tags" => [
        [0] "_dateparsefailure"
    ],
       "message" => "2014-10-05 02:59:25",
    "@timestamp" => 2019-11-11T04:07:37.236Z
}

Working

2015-10-05 02:59:25
{
      "@version" => "1",
          "host" => "elastic18",
    "@timestamp" => 2015-10-04T15:59:25.000Z,
       "logdate" => "2015-10-05 02:59:25",
       "message" => "2015-10-05 02:59:25"
}

Thank you

Under your local DST rules are you sure that 2015-10-05 02:59:25 existed?

@Badger

Sorry I don't understand what you mean by DST rules, are you talking about the ELK server?

My ELK server runs on Linux and the timezone is set to Australia/Melbourne

zdump -v Australia/Melbourne | grep 2015
Australia/Melbourne  Sat Apr  4 15:59:59 2015 UT = Sun Apr  5 02:59:59 2015 AEDT isdst=1 gmtoff=39600
Australia/Melbourne  Sat Apr  4 16:00:00 2015 UT = Sun Apr  5 02:00:00 2015 AEST isdst=0 gmtoff=36000
Australia/Melbourne  Sat Oct  3 15:59:59 2015 UT = Sun Oct  4 01:59:59 2015 AEST isdst=0 gmtoff=36000
Australia/Melbourne  Sat Oct  3 16:00:00 2015 UT = Sun Oct  4 03:00:00 2015 AEDT isdst=1 gmtoff=39600

If I change the date filter timezone to +10 instead of Australia/Melbourne, it seems to be working fine but its out by an hour

logdate	2019-11-11 23:14:40
@timestamp	Nov 12, 2019 @ 00:14:40.000

EDIT: temporary brain freeze, It should be +11 :blush:

I still don't understand why the original timezone doesn't work

No, it is not an ELK thing. DST is about the time suddenly jumping from 2:00 to 3:00 (meaning 2:59 does not exist) once in spring and then backwards in the autumn (so 2:01 happens twice, an hour apart). Right now I just feel too lazy to work out if clocks go back or forward in the Southern hemisphere in the Autumn.

Thanks for the insight

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.