Date parsing [5.6.2]

[cperriot (Cedric Perriot)]

trying to parse:

2016-03-27 01:45:00
2016-02-20 02:15:00
2017-10-10 09:57:46

with the following config:

input {
file {
sincedb_path=>"sincedb"
path => "/xxxxxxx/export_test.txt"
start_position => "beginning"
}
}

filter {
csv {
columns => ["start"]
separator => ","
}

date {
locale => "en"
match => ["start", "YYYY-MM-dd HH:mm:ss"]
timezone => "Europe/London"
target => "@timestamp"
}

the result is :

{
"duration" => 2,
"path" => "/users/cperriot/logstash-5.6.2/bin/export_test.txt",
"@timestamp" => 2017-10-10T12:55:18.085Z,
"@version" => "1",
"host" => "xxx.fr.intranet",
"start" => "2016-03-27 01:45:00",
"message" => "2016-03-27 01:45:00\r",
"tags" => [
[0] "_dateparsefailure"
]
}
{
"duration" => 3,
"path" => "/users/cperriot/logstash-5.6.2/bin/export_test.txt",
"@timestamp" => 2016-02-20T10:15:00.000Z,
"@version" => "1",
"host" => "xxx.fr.intranet",
"start" => "2016-02-20 10:15:00",
"message" => "2016-02-20 10:15:00\r",
"status" => "16"
}
{
"duration" => 269,
"path" => "/users/cperriot/logstash-5.6.2/bin/export_test.txt",
"@timestamp" => 2017-10-10T08:57:46.000Z,
"@version" => "1",
"host" => "xxx.fr.intranet",
"start" => "2017-10-10 09:57:46",
"message" => "2017-10-10 09:57:46\r",
"status" => "16"
}

There are 2 things I don't understand,

1. why the parsing failure, it seems to be linked to the timezone offset but I don't get it.

2. Why one date stays the same 10:15 whereas the other get offsetted by an hour , 9:57 to 8:57 ?

thank you

[cperriot (Cedric Perriot)]

OK I think I understood the second point: It is taking the date into account to know about winter/summer time.
am i right ?

[magnusbaeck (Magnus Bäck)]

why the parsing failure, it seems to be linked to the timezone offset but I don't get it.

I don't know, but when the date filter fails it will log a message that typically points to the part of the string it has a problem with.

OK I think I understood the second point: It is taking the date into account to know about winter/summer time.
am i right ?

Yes.

[Badger]

@cperriot

2016-03-27 01:45:00

That time is not valid. BST started at 2016-03-27 01:00:00, we skipped straight to 2016-03-27 02:00:00