# Failed to Parse Date Field

**URL:** <https://discuss.elastic.co/t/failed-to-parse-date-field/284576>\
**Category:** Logstash\
**Created:** [September 19, 2021, 4:51pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/284576 "2021-09-19T16:51:24Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tim\_Mobley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_mobley/32/94741_2.png) [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\
**Post date:** [September 19, 2021, 4:51pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/284576/1 "2021-09-19T16:51:24Z")

</div>

I have some CSV files I'm trying to ingest using Logstash's [CVS filter plugin](https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html). I'm needing the "Date" column to represent the event time field in Kibana (rather than @timestamp, which just tells me when I imported the CSV). In the CSV file, the date column is formatted as follows:  
Date  
9/17/2021  
9/16/2021  
9/15/2021  
9/15/2021  
9/14/2021  
9/13/2021  
9/13/2021  
9/13/2021  
9/7/2021  
9/7/2021  
9/2/2021  
9/1/2021  
9/1/2021  
9/1/2021  
8/31/2021

My logstash filter block looks like this:

```auto
filter {
  csv {
    columns => ["Date", "Description", "Withdrawals", "Deposits", "Balance"]
  }
  mutate {
    gsub => [
      "Withdrawals", "[$,]", "",
	  "Deposits", "[$,]", "",
	  "Balance", "[$,]", ""
    ]
  }
}

```

...and I have an index template with the following mapping:

```auto
{
  "properties": {
    "Deposits": {
      "scaling_factor": 100,
      "type": "scaled_float"
    },
    "Withdrawals": {
      "scaling_factor": 100,
      "type": "scaled_float"
    },
    "Balance": {
      "scaling_factor": 100,
      "type": "scaled_float"
    },
    "Date": {
      "type": "date"
    }
  }
}

```

...when I do not map the Date field as date, it is imported as text and not available as the definitive time field. However when I use the template above, I get the following WARNING:

> [2021-09-19T12:32:55,126][WARN][logstash.outputs.elasticsearch][pipeline-name][xxxxxxxxxxxxxx] Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"pipeline-name-2021.09.19", :routing=\>nil}, {"Balance"=\>"10.00", "host"=\>"ComputerName", "message"=\>""06/28/2021","DETAILS"\r", "Date"=\>"06/28/2021", "Withdrawals"=\>"10.00", "Deposits"=\>"", "path"=\>"C:/Path/To/File.csv", "@timestamp"=\>2021-09-19T16:32:50.236Z, "Description"=\>"DESCRIPTION", "Category"=\>"category", "@version"=\>"1"}], :response=\>{"index"=\>{"\_index"=\>"index-name-2021.09.19", "\_type"=\>"\_doc", "\_id"=\>"xxxxxxxxxxxxx", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field [Date] of type [date] in document with id 'xxxxxxxxxxx'. Preview of field's value: '06/28/2021'", "caused\_by"=\>{"type"=\>"illegal\_argument\_exception", "reason"=\>"failed to parse date field [06/28/2021] with format [strict\_date\_optional\_time||epoch\_millis]", "caused\_by"=\>{"type"=\>"date\_time\_parse\_exception", "reason"=\>"Failed to parse with all enclosed parsers"}}}}}}

I have tried using the "convert" setting in the CSV filter plugin to identify the "Date" field as a "date" type, but that yield the same outcome.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [September 19, 2021, 5:14pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/284576/2 "2021-09-19T17:14:37Z")

</div>

You could parse the field using a date filter to change the [date] field into a LogStash::Timestamp object. That will be sent to elasticsearch as a number of milliseconds since the epoch.

> "failed to parse date field [06/28/2021] with format [strict\_date\_optional\_time||epoch\_millis]"

The default parsers for a field configured as a "date" in elasticsearch can handle milliseconds since the epoch.

If you do not want to do the conversion in logstash then [configure a date parser](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html) as part of your elasticsearch mapping. I think it would be

```
"Date": {
  "type": "date",
  "format": "M/d/y"
}

```

You would need to read up on [DateTimeFormatter](https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html) to decide whether you might want u or Y instead of y and whether you want to use yyyy instead of y.

---

<div class="post-metadata">

**Author:** ![Tim\_Mobley](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tim_mobley/32/94741_2.png) [@Tim\_Mobley](https://discuss.elastic.co/u/Tim_Mobley)\
**Post date:** [September 19, 2021, 10:34pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/284576/3 "2021-09-19T22:34:18Z")

</div>

Thank you so much, @Badger - that did the trick!

I added the **date** filter plugin as you recommended. However, the default target is **@timestamp** , which resulted in every row of the csv generating its own index - not what I intended. So I did the following instead:

```auto
date {
    match => ["Date", "M/d/yyyy"]
    target => "Date"
  }

```

...then set chose the **Date** field instead of **@timestamp** when creating the index pattern. This worked perfectly, so thank you again for pointing me in the right direction.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 17, 2021, 10:34pm UTC](https://discuss.elastic.co/t/failed-to-parse-date-field/284576/4 "2021-10-17T22:34:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.