# Csv filter convert to date doesn't work

**URL:** <https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609>\
**Category:** Logstash\
**Created:** [October 27, 2017, 7:55pm UTC](https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609 "2017-10-27T19:55:19Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![szgaljic](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@szgaljic](https://discuss.elastic.co/u/szgaljic)\
**Post date:** [October 27, 2017, 7:55pm UTC](https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609/1 "2017-10-27T19:55:19Z")

</div>

I have a column called **Birthday** that is formatted as **dd/MM/yyyy** (i.e., 3/8/1983) and would like Logstash to treat this as a date when passing to Elasticsearch so the mapping is auto-generated as a **date**.

This does not work - Elasticsearch is generating the mapping as text.

> filter {  
> csv {  
> columns =\> [  
> "Age",  
> "Birthday",  
> "College",  
> "Name",  
> "Weight"  
> ]  
> convert =\> {  
> "Age" =\> "integer"  
> "Birthday" =\> "date"  
> "Weight" =\> "integer"  
> }  
> }  
> }

Elasticsearch generates the mapping for this field like this:

> "Birthday": {  
> "fields": {  
> "keyword": {  
> "ignore\_above": 256,  
> "type": "keyword"  
> }  
> },  
> "type": "text"  
> },

I also tried to use the mutate filter but that does not work either, it's still mapped as text.

What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 30, 2017, 6:46am UTC](https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609/2 "2017-10-30T06:46:30Z")

</div>

What does an example document look like?

---

<div class="post-metadata">

**Author:** ![szgaljic](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@szgaljic](https://discuss.elastic.co/u/szgaljic)\
**Post date:** [October 30, 2017, 10:14pm UTC](https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609/3 "2017-10-30T22:14:46Z")

</div>

HI Magnus,

The actual setup can be found in my [elasticsearch-injest-logstash](https://github.com/JahnelGroup/docker-compose/tree/master/elasticsearch-injest-logstash) Github project. I'm specifically loading the only CSV file found in the [csv directory](https://github.com/JahnelGroup/docker-compose/tree/master/elasticsearch-injest-logstash/csv).

I simplified my question for this forum and reduced the number of actual columns so a sample row could look like this:

```
56,9/11/1960,Louisiana Tech,"Fowler, Bobby",230
30,9/30/1986,LSU,"Johnson, Quinn",255
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 31, 2017, 3:20am UTC](https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609/4 "2017-10-31T03:20:14Z")

</div>

You could use a date filter on that field, I can't comment on why it's not working though sorry.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 31, 2017, 6:04am UTC](https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609/5 "2017-10-31T06:04:18Z")

</div>

I meant what a document going _out_ of Logstash looked like but I was able to reconstruct it:

```nohighlight
$ echo '56,9/11/1960,Louisiana Tech,"Fowler, Bobby",230' | /opt/logstash/bin/logstash -f test.config
Settings: Default pipeline workers: 8
Pipeline main started
{
       "message" => "56,9/11/1960,Louisiana Tech,\"Fowler, Bobby\",230",
      "@version" => "1",
    "@timestamp" => "2017-10-31T06:00:12.150Z",
          "host" => "lnxolofon",
           "Age" => 56,
      "Birthday" => "9/11/1960",
       "College" => "Louisiana Tech",
          "Name" => "Fowler, Bobby",
        "Weight" => 230
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}

```

I'm not sure how the csv filter's date conversion is supposed to work but it doesn't appear to be doing anything for the date. As Mark says you can use a date filter to convert it to an ISO8601 timestamp that ES will recognize out of the box but you could also adjust the index mappings so that it treats mm/dd/yyyy strings as dates.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 28, 2017, 6:04am UTC](https://discuss.elastic.co/t/csv-filter-convert-to-date-doesnt-work/105609/6 "2017-11-28T06:04:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.