# Change the format of dates & calculate the difference

**URL:** <https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899>\
**Category:** Logstash\
**Created:** [July 7, 2016, 5:12am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899 "2016-07-07T05:12:55Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ravi\_Shanker\_Reddy](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@Ravi\_Shanker\_Reddy](https://discuss.elastic.co/u/Ravi_Shanker_Reddy)\
**Post date:** [July 7, 2016, 5:12am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899/1 "2016-07-07T05:12:55Z")

</div>

I am having a csv. In that I am having two times like this  
Wed May 18 23:00:01.000 2016 | Wed May 18 23:01:32.236 2016

I want to change this as date format and find the difference between them. I tried like this. But while running ending up with errors but with configtest it says ok.

`filter { csv {`  
`columns => ["CDRtime","RcdNo","OrigNo","DestNo","MST","MDT","SrvTyp","DelvSts","Prio","RcdTyp"]`  
`separator => "|" }`  
`date { match => ["MST", "UNIX"] #request submittion time target => "MST" } }`

Errors:

`Failed parsing date from field {:field=>"MST", :value=>"Wed May 18 23:00:01 2016", :exception=>"Invalid UNIX epoch value 'Wed May 18 23:00:01 2016'", :config_parsers=>"UNIX", :config_locale=>"default=en_US", :level=>:warn}`

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 7, 2016, 5:40am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899/2 "2016-07-07T05:40:08Z")

</div>

Use a date format pattern that matches your input (the UNIX pattern isn't the right choice here). Consult the date filter documentation.

---

<div class="post-metadata">

**Author:** ![Ravi\_Shanker\_Reddy](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@Ravi\_Shanker\_Reddy](https://discuss.elastic.co/u/Ravi_Shanker_Reddy)\
**Post date:** [July 7, 2016, 5:44am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899/3 "2016-07-07T05:44:27Z")

</div>

I checked all the patterns here [https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns](https://github.com/elastic/logstash/blob/v1.4.2/patterns/grok-patterns). But nothing suits me.

I tried like this now  
`match => ["MST", "DAY MONTH ([1-9]{2}|\s[1-9]|10) TIME YEAR" ]`

This also not working. How to match this

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 7, 2016, 5:55am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899/4 "2016-07-07T05:55:56Z")

</div>

You're looking for a date pattern, not a grok pattern. See [https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-match](https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-match) and follow the [joda.time.format.DateTimeFormat](http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html) link.

---

<div class="post-metadata">

**Author:** ![Ravi\_Shanker\_Reddy](https://avatars.discourse-cdn.com/v4/letter/r/a5b964/32.png) [@Ravi\_Shanker\_Reddy](https://discuss.elastic.co/u/Ravi_Shanker_Reddy)\
**Post date:** [July 7, 2016, 6:32am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899/5 "2016-07-07T06:32:41Z")

</div>

`date { match => ["MST", "EEE MMM dd HH:mm:ss.SSS YYYY", "EEE MMM d HH:mm:ss.SSS YYYY"] target => "MST" } date { match => ["MDT", "EEE MMM dd HH:mm:ss.SSS YYYY", "EEE MMM d HH:mm:ss.SSS YYYY"] target => "MDT" }`

Its solves my problem but I am seeing these fields as a string in Kibana. But I need that fields as timestamps.

For that how can I map the fields in ES. From this link  
[https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html)

But in my case how to write this mappings.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [July 7, 2016, 7:19am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899/6 "2016-07-07T07:19:18Z")

</div>

Elasticsearch should autodetect the MDT field as a date, but since you've previously indexed the field as a string that'll stick. The mapping of a field can't be changed without reindexing. But, as you presumably are using a time-series index this should correct itself until tomorrow.

To make sure a field is mapped a certain way you can modify the default index template used for your indexes.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:49am UTC](https://discuss.elastic.co/t/change-the-format-of-dates-calculate-the-difference/54899/7 "2017-07-06T04:49:10Z")

</div>


