# Uploading csv file: Failed to parse date field dd/MM/yyyy HH:mm **URL:** \ **Category:** Logstash\ **Created:** [April 21, 2021, 9:23am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813 "2021-04-21T09:23:19Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 9:23am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/1 "2021-04-21T09:23:20Z")
Hi, I'm uploading a csv file (; as separator). I identified one of the columns as Machine Time. The format on excel is "customized" as seen in the image below: ![immagine](https://us1.discourse-cdn.com/elastic/original/3X/2/f/2f60908beeae46ef3526a91a592741b2502585ae.png) To parse the date as a date type on Elastic I used the date filter: ```auto date { match => ["Machine Time", "dd/MM/yyyy HH:mm", "ISO8601", "dd/MM/yyyy HH:mm:ss"] target => "Machine Time" } ``` The strange thing is that on for example 6000 lines, it loads 5995 correctly and the missing 5 do not, even if the format is the same for all of them. On the logstash logs I see the following error: Preview of field's value: '28/03/2021 02:05' This is one of the 5 dates it fails to upload. Can anyone help me? What can i do to fix? Thanks. Marco ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [April 21, 2021, 9:51am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/2 "2021-04-21T09:51:02Z")
Looks like there's a space after the day. So to catch that then add a space in your match also. ```auto date { match => ["Machine Time", "dd/MM/yyyy HH:mm", "dd /MM/yyyy HH:mm", "ISO8601", "dd/MM/yyyy HH:mm:ss"] target => "Machine Time" } ``` ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 9:55am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/3 "2021-04-21T09:55:20Z")
Hi aaron, I don't know why I copied the string like this but there is no space. Now I edit the post. ![immagine](https://us1.discourse-cdn.com/elastic/original/3X/e/3/e3ed51c8f8031b15629ad0b2aaae1a982a7925fb.png) Marco ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [April 21, 2021, 10:05am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/4 "2021-04-21T10:05:09Z")
The date alone parses correctly so I wouldn't think this has anything to do with the date. Is there more logic in your configuration that could be it? Are you able to post your .conf? ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 10:16am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/5 "2021-04-21T10:16:15Z")
```auto input { file { path => "PATH/file_name.csv" start_position => "beginning" sincedb_path => PATH } } filter { csv { columns => [" ***", "Machine Time", "***", "***", ..., other 140 columns name] separator => ";" "***" => "integer" (this for each number column, almost 40 columns) date { match => ["Machine Time", "dd/MM/yyyy HH:mm", "ISO8601", "dd/MM/yyyy HH:mm:ss"] target => "Machine Time" } } output { stdout { codec => rubydebug } elasticsearch { hosts => ["localhost:9200"] index => "index_name" user => "***" password => "***" } } ``` ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [April 21, 2021, 10:29am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/6 "2021-04-21T10:29:02Z")
Nothing looks out of place. Are you able to isolate the **same** 5 records each time? Have you looked at the file with a text editor and not within Excel to verify no extra/special characters that could be causing it? ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 10:43am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/7 "2021-04-21T10:43:45Z")
To make you understand the situation a little. I'm doing a test with a csv populated with data coming from a mysql database. Probably in the future I will connect directly to the database, but for now I want to use this csv and populate it manually. It is therefore a continuous flow of data ... I cannot afford to have this problem repeat itself again. If it happens with the first 6000 lines, it will probably happen with the new data. So the solution is not to isolate these 5 lines but to solve the problem, because otherwise it will repeat itself with the new data without my understanding the real cause. Anyway, by block note i see that (so no extra caracters): ![immagine](https://us1.discourse-cdn.com/elastic/original/3X/e/6/e6a891e74c397c3b7fb1cfa7a2cf54cf36071117.png) ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [April 21, 2021, 10:47am UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/8 "2021-04-21T10:47:44Z")
I'm stumped from what I am seeing. Are you able to share the CSV to see if I can replicate the results? ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 12:28pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/9 "2021-04-21T12:28:36Z")
How do I share a csv file to you? ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [April 21, 2021, 12:39pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/10 "2021-04-21T12:39:55Z")
[https://pastebin.com/](https://pastebin.com/) or [https://gist.github.com/](https://gist.github.com/) would probably be the easiest. ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 12:54pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/11 "2021-04-21T12:54:33Z")
> ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 12:57pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/12 "2021-04-21T12:57:49Z")
I had to omit some string format columns because they are sensitive data ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [April 21, 2021, 1:16pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/13 "2021-04-21T13:16:52Z")
I am not sure with what was given. 🤷‍♂️ The CSV has 13102 and it ingested them all with a correct date conversion. Can you post the full date parse error in the log? ```auto { "count" : 13102, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ``` ```auto Machine Time" : "2021-04-19T13:56:00.000Z", ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 21, 2021, 1:26pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/14 "2021-04-21T13:26:49Z")
> [@MARCO\_RAMBALDI](#): > > '28/03/2021 02:05' What time zone are you in? Did 02:05 exist in that time zone, or did the time go from 01:59:59 to 03:00:00 and skip over the 02 hour? ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 1:29pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/15 "2021-04-21T13:29:48Z")
How is it possible? The file I shared with you has 6551 raws, not 13102. How did you get that output you show me? I'm afraid the file was truncated or modified in some ways when I uploaded it to github gist. How many raws do you see? ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 1:30pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/16 "2021-04-21T13:30:37Z")
I am in Italy. How do I answer your question? Where do I see it exactly? In my advanced settings, i see that Timezone depends to Browser. ---
**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\ **Post date:** [April 21, 2021, 1:36pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/17 "2021-04-21T13:36:33Z")
The file at [failed parse error file · GitHub](https://gist.github.com/marcorambaldi/b1dde3a911a9de9ec21fcc8a56524c67) has 3513 lines. Not sure how I got that many results before since I downloaded the file and that's what was there. This time I just copy/pasted the rows. But reran that one and got the same results. ```auto { "count" : 3513, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ``` ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 21, 2021, 1:38pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/18 "2021-04-21T13:38:45Z")
> [@MARCO\_RAMBALDI](#): > > I am in Italy. Then you are on CEDT, which starts on the last Sunday in March. 2 AM on March 28th did not happen, the time skipped to 3 AM, so a date filter cannot parse it. More commentary [here](https://discuss.elastic.co/t/logstash-date-function-mapping-timestamp-ends-with-error-dateparsefailure-when-a-hour-is-02/227679/5). ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 1:39pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/19 "2021-04-21T13:39:31Z")
> [@aaron-nimocks](#): > > The file at [failed parse error file · GitHub](https://gist.github.com/marcorambaldi/b1dde3a911a9de9ec21fcc8a56524c67) has 3513 lines. Not sure how I got that many results before since I downloaded the file and that's what was there. This time I just copy/pasted the rows. > > But reran that one and got the same results. You had no problems with parse error because the lines that fail to load were truncated when I uploaded the file to github. ---
**Author:** ![MARCO\_RAMBALDI](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_rambaldi/32/46108_2.png) [@MARCO\_RAMBALDI](https://discuss.elastic.co/u/MARCO_RAMBALDI)\ **Post date:** [April 21, 2021, 1:43pm UTC](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813/20 "2021-04-21T13:43:09Z")
I risked taking the computer and throwing it out the window ... 🤣 I've been trying to figure out the problem for three weeks [Next page](https://discuss.elastic.co/t/uploading-csv-file-failed-to-parse-date-field-dd-mm-yyyy-hh-mm/270813.md?page=2)