Mysterious _dateparsefailure on identical fields

Elastic Stack Logstash
1

I have included below 3 csv input lines that are virtually identical to one another. The date field in the first and last parse properly whereas I get a _dateparsefailure on the middle line.

The logstash date field is shown below.
date {
match => [ "timestamp", "ddMMMyy:H", "ddMMMyy:HH" ]
target => "@timestamp"
}

Line which starts with "1379" is the one that fails.

"1378","Commercial","eManPortl","CB_AX01A","MDB-Cntr","mdb DAATradeDoc","08MAR15",1,5,0.00,0.00,0.01,1.72,6.73,0.03,0.00011,0.00,0.03,0.00,0.00,0.00,0.00,0.00,0.070441,0.056069,".",".",0.145431,0.000545,0.000473,0.143660,0.000753,0.01,8.45

"1379","Commercial","eManPortl","CB_AX01A","MDB-Cntr","mdb DAATradeDoc","08MAR15",2,3,0.00,0.00,0.01,1.97,7.51,0.04,0.00013,0.00,0.04,0.00,0.00,0.00,0.00,0.00,0.047428,0.037567,".",".",0.108691,0.000377,0.000471,0.107275,0.000568,0.01,9.49

"1380","Commercial","eManPortl","CB_AX01A","MDB-Cntr","mdb DAATradeDoc","08MAR15",3,1,0.00,0.00,0.00,1.96,8.19,0.05,0.00013,0.00,0.05,0.00,0.00,0.00,0.00,0.00,0.016913,0.013649,".",".",0.048059,0.000126,0.000127,0.047573,0.000233,0.00,10.15

If anyone can identify the reason for the failure it would be highly appreciated.

Pete Gervais

2

Is the hour part 2 AM? That did not exist on 2015/3/8. DST started and we went straight from 1:59:59 to 3:00:00. See here for a possible solution.

3

I have many lines in the csv file that has similar hour input which dont fail.
I'm using logstash 7.4.0.

4

Are you asserting that "08MAR15:2" will sometimes be parsed and sometimes get a _dateparsefailure?

5

Badger,
I have used your answer on selectively printing errors to run through all my input.
Now I only see the errors and low and behold, all failures are on a date field that looks :
"08MAR15",2 where hour == 2.

Does version 7.8.0 of logstash solve this issue out of the box or must I still do some special processing?

6

There is nothing elastic can do to fix that. That time did not occur that day. You need to decide what you want to do with times for that day (which only had 23 hours). You may also need to think about November 1, 2015, which had 25 hours. Depending on your exact use case that may or may not matter.

1 Like
7

Badger,
As additional data , I have installed Version 7.8.0 and I still get the same error.
There are many entries that contain the second hour of the i.e. 2 but it only happen when the date is "08MAR15".

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.