Logstash date not matching

Elastic Stack Logstash
1

Hey all,
I'm trying to work out why I get a grok parse failure on date here when using these two as input

2014-10-05 02:59:25 Works
2015-10-05 02:59:25 Does not work

timestamp.conf

   input {
        stdin { }
    }


    filter {
     grok {
        match => { "message" => "%{TIMESTAMP_ISO8601:logdate}" }
     }}

filter {
	date {
	match => [ "logdate", "yyyy-MM-dd HH:mm:ss" ]
	target => "@timestamp"
}}
output {
    stdout { codec => rubydebug }
}

./logstash -f timestamp.conf -e

Not Working

2014-10-05 02:59:25
{
      "@version" => "1",
          "host" => "elastic18",
       "logdate" => "2014-10-05 02:59:25",
          "tags" => [
        [0] "_dateparsefailure"
    ],
       "message" => "2014-10-05 02:59:25",
    "@timestamp" => 2019-11-11T04:07:37.236Z
}

Working

2015-10-05 02:59:25
{
      "@version" => "1",
          "host" => "elastic18",
    "@timestamp" => 2015-10-04T15:59:25.000Z,
       "logdate" => "2015-10-05 02:59:25",
       "message" => "2015-10-05 02:59:25"
}

Thank you

2

Under your local DST rules are you sure that 2015-10-05 02:59:25 existed?

3

@Badger

Sorry I don't understand what you mean by DST rules, are you talking about the ELK server?

My ELK server runs on Linux and the timezone is set to Australia/Melbourne

zdump -v Australia/Melbourne | grep 2015
Australia/Melbourne  Sat Apr  4 15:59:59 2015 UT = Sun Apr  5 02:59:59 2015 AEDT isdst=1 gmtoff=39600
Australia/Melbourne  Sat Apr  4 16:00:00 2015 UT = Sun Apr  5 02:00:00 2015 AEST isdst=0 gmtoff=36000
Australia/Melbourne  Sat Oct  3 15:59:59 2015 UT = Sun Oct  4 01:59:59 2015 AEST isdst=0 gmtoff=36000
Australia/Melbourne  Sat Oct  3 16:00:00 2015 UT = Sun Oct  4 03:00:00 2015 AEDT isdst=1 gmtoff=39600

If I change the date filter timezone to +10 instead of Australia/Melbourne, it seems to be working fine but its out by an hour

logdate	2019-11-11 23:14:40
@timestamp	Nov 12, 2019 @ 00:14:40.000

EDIT: temporary brain freeze, It should be +11 :blush:

I still don't understand why the original timezone doesn't work

4

No, it is not an ELK thing. DST is about the time suddenly jumping from 2:00 to 3:00 (meaning 2:59 does not exist) once in spring and then backwards in the autumn (so 2:01 happens twice, an hour apart). Right now I just feel too lazy to work out if clocks go back or forward in the Southern hemisphere in the Autumn.

5

Thanks for the insight

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.