# Date parse error on only one record **URL:** \ **Category:** Logstash\ **Created:** [October 29, 2020, 7:36am UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653 "2020-10-29T07:36:19Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Evan\_Vujcec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evan_vujcec/32/78034_2.png) [@Evan\_Vujcec](https://discuss.elastic.co/u/Evan_Vujcec)\ **Post date:** [October 29, 2020, 7:36am UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/1 "2020-10-29T07:36:19Z")
I'm trying to import data from a csv into an ES index using logstash. In the CSV I have seperate date and time columns that I'm combining and then parsing it with the date filter plugin so it can use it as the @timestamp. There are 150500 records and all of pass and are correctly matched except for 1 record. Reviewing this record there is nothing obviously abnormal about it that would necessarily cause such an issue so I'm at a loss. I've tried deleting the index and rerunning logstash multiple times and each time the same record fails. The record is tagged with the `_dateparsefailure` tag and its @timestamp is the only one containing the upload time instead of the parsed. I'm new to logstash so there's probably a better way to do this but I have a field called date that contains a "Date" like so "MM/dd/yyyy 12:00:00 AM" (yes every record is 12am) and a "Time" field like so "HH:mm". I pass the following filters: ```auto truncate { fields => "Date" length_bytes => 10 } mutate { add_field => { "DateTime" => "%{Date} %{Time}"} remove_field => ["Date", "Time"] } date { match => ["DateTime", "MM/dd/yyyy HH:mm"] timezone => "America/Los_Angeles" remove_field => ["DateTime"] } ``` The record in question contains Date and Time like so `03/13/2016 12:00:00 AM,02:30`. I should also point out that I seem to get no error if I also output to stdout, it just shows the tags with `_dateparsefailure`. ---
**Author:** ![Wolfram\_Haussig](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wolfram_haussig/32/70528_2.png) [@Wolfram\_Haussig](https://discuss.elastic.co/u/Wolfram_Haussig)\ **Post date:** [October 29, 2020, 7:59am UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/2 "2020-10-29T07:59:36Z")
Hi, I guess that the date filter fails because of the suffix "` AM,02:30`". Although I have not tried it I think the correct format definition would be(see [here](https://www.joda.org/joda-time/key_format.html) for details): `MM/dd/yyyy HH:mm a,ZZ`. You can provide multiple formats for the date filter so LogStash tries both and chooses the correct one: ```auto date { match => ["DateTime", "MM/dd/yyyy HH:mm", "MM/dd/yyyy HH:mm a,ZZ"] timezone => "America/Los_Angeles" remove_field => ["DateTime"] } ``` Best regards Wolfram ---
**Author:** ![Evan\_Vujcec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evan_vujcec/32/78034_2.png) [@Evan\_Vujcec](https://discuss.elastic.co/u/Evan_Vujcec)\ **Post date:** [October 29, 2020, 4:00pm UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/3 "2020-10-29T16:00:55Z")
So because I truncate the Date field to 10 bytes the `12:00:00 AM` is removed before DateTime is even created. The DateTime field comes out to this `"DateTime" => "03/13/2016 02:30"` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 29, 2020, 4:09pm UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/4 "2020-10-29T16:09:40Z")
> [@Evan\_Vujcec](#): > > `03/13/2016 12:00:00 AM,02:30` . That time is not valid if you are on Eastern time. The time went directly from 02:00:00 to 03:00:01 because daylight savings began. ---
**Author:** ![Evan\_Vujcec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evan_vujcec/32/78034_2.png) [@Evan\_Vujcec](https://discuss.elastic.co/u/Evan_Vujcec)\ **Post date:** [October 29, 2020, 4:35pm UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/5 "2020-10-29T16:35:06Z")
I have plenty of other records that occur at similar times like `03/12/2016 12:00:00 AM,02:45,` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 29, 2020, 4:38pm UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/6 "2020-10-29T16:38:54Z")
Daylight savings did not start on the 12th, it started on the 13th. So only on the 13th did times between 2 and 3 AM not exist. ---
**Author:** ![Evan\_Vujcec](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/evan_vujcec/32/78034_2.png) [@Evan\_Vujcec](https://discuss.elastic.co/u/Evan_Vujcec)\ **Post date:** [October 29, 2020, 5:18pm UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/7 "2020-10-29T17:18:25Z")
Oh gotcha that makes sense. I'm assuming it was just a data entry error then. There are plenty of records during 01 and 03 on the 13th and this was the only record for 02. Is there a better way of combining the Date and Time fields just in the date filter so I don't have to pass through the truncate and mutate filters? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [October 29, 2020, 5:19pm UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/8 "2020-10-29T17:19:25Z")
> [@Evan\_Vujcec](#): > > Is there a better way of combining the Date and Time fields just in the date filter so I don't have to pass through the truncate and mutate filters? Not that I can think of. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 26, 2020, 5:19pm UTC](https://discuss.elastic.co/t/date-parse-error-on-only-one-record/253653/9 "2020-11-26T17:19:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.