# How to deal with the transition when a field changes format "string" -\> "timestamp"

**URL:** <https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649>\
**Category:** Elasticsearch\
**Created:** [August 3, 2019, 6:33pm UTC](https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649 "2019-08-03T18:33:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\
**Post date:** [August 3, 2019, 6:33pm UTC](https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649/1 "2019-08-03T18:33:18Z")

</div>

Hello.

Facts:

- Using ElasticSearch version **5.6.10**

- Currently, the Template [1] for one of my indices contains a section like this, since **LogStash is creating field "_my\_datefield_" as a string** , with one of these two patterns.

- I need to change the way LogStash builds "_my\_datefield_", to use the Filter plugin Date. Therefore, **the type of "_my\_datefield_" won't be anymore a "string" but a "LogStash::Timestamp" type.**"_my\_datefield_" will look like this

- I assume I can create a new Template [2] to address the new type, like this

- My understanding is that the **new Template [2] only will affect new indices**. Correct? As they are created daily, "_my\_index-yyyy.mm.dd_", that means **Template [2] will start being used the day after I set it up**, right?

So **my question is how to allow ElasticSearch to accept both types** , string and timestamp, during the day of the transition. The day I set the new code for LogStash, the index for that day still uses Template [1], but "_my\_datefield_" won't be a string anymore. My experiments tell me that ElasticSearch will reject the new documents as it does not know how parse the field "_my\_datefield_" anymore. Is there a solution for this?

Thanks a lot in advance.

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 5, 2019, 10:07am UTC](https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649/2 "2019-08-05T10:07:09Z")

</div>

you are right regarding the template, it will only kick into effect once a new index is created. However, due to sending the date as a string, you will be able to send the date as a string on day X (and it will be indexed as a string) and as a string on day X+1 and it will be indexed as a date.

One thing that could happen however is, that documents start to get rejected, because the date is not in the required format.

---

<div class="post-metadata">

**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\
**Post date:** [August 5, 2019, 1:56pm UTC](https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649/3 "2019-08-05T13:56:25Z")

</div>

I have changed the Template, adding a new format to the property, like this

```
"my_datefield": {
    "type": "date",
    "format": "MM/dd/yy HH:mm:ss||MM/dd/yy HH:mm:ss.SSS||MM/dd/yy HH:mm:ss.1000||yyyy-MM-dd'"'T'"'HH:mm:ss.SSS'"'Z'"'"
}

```

The ' " ' part is to escape the single quotes around T and Z, as I upload the template entirely from the command line.

So, **will adding yyyy-MM-dd'T'HH:mm:ss.SSS'Z' to the format make ElasticSearch to accept documents when date fields become Logstash::timestamp instead of string?**

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [August 6, 2019, 7:56am UTC](https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649/4 "2019-08-06T07:56:46Z")

</div>

there is no such thing as a logstash timestamp for Elasticsearch. A value is received as a string and tried to be converted.

--Alex

---

<div class="post-metadata">

**Author:** ![jcaballero](https://avatars.discourse-cdn.com/v4/letter/j/278dde/32.png) [@jcaballero](https://discuss.elastic.co/u/jcaballero)\
**Post date:** [August 6, 2019, 7:28pm UTC](https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649/5 "2019-08-06T19:28:42Z")

</div>

yeah, you are right.  
Adding "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" did the trick 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 3, 2019, 7:44pm UTC](https://discuss.elastic.co/t/how-to-deal-with-the-transition-when-a-field-changes-format-string-timestamp/193649/6 "2019-09-03T19:44:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.