# Convert field containing a timestamp from string to a date / time or timestamp?

**URL:** <https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594>\
**Category:** Logstash\
**Created:** [February 26, 2021, 10:26am UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594 "2021-02-26T10:26:02Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 2, 2021, 3:44pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/21 "2021-03-02T15:44:55Z")

</div>

But Im confused on where do I define this.

Logstash? Elasticsearch? If Elasticsearch, where?

etc

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 2, 2021, 3:54pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/22 "2021-03-02T15:54:35Z")

</div>

In Kibana or using Curl. Below is a Curl example.

```auto
curl -X PUT "localhost:9200/my-index-000001/_mapping?pretty" -H 'Content-Type: application/json' -d'
{
  "properties": {
    "email": {
      "type": "keyword"
    }
  }
}
'

```

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 2, 2021, 3:57pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/23 "2021-03-02T15:57:37Z")

</div>

It would be Kibana.

When I setup a Kibana Index, it automatically detects it. Are you saying I need to "force it"?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 2, 2021, 3:59pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/24 "2021-03-02T15:59:47Z")

</div>

You shouldn't have to force it if your data is coming in like you shown above. But it seems you still have something in your index or pattern that is setting it as a text/keyword.

The best practice is to set the mapping yourself so the data is mapped the way you want it. If you don't then Elastic will guess what the type is and isn't always what you want.

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 2, 2021, 4:09pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/25 "2021-03-02T16:09:42Z")

</div>

OK but I want to make it clear: When you mention that I set the mapping myself, where exactly do I set that up? That is what Im confused about.

When I setup a Kibana Index, I set up to watch "myindex-data-\*" which gets it from myindex-data-20210302, myindex-data-20210301, etc...

All my data is recieved to Logstash THEN it gets sent to Elasticsearch. Elasticsearch doesnt recieve data directly.

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 2, 2021, 4:36pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/26 "2021-03-02T16:36:59Z")

</div>

I apologize. I can see where I am confusing you since you are using dynamically created index and not static.

You would create an [index template](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html) in this case and include the pattern. Then anything that matches that pattern that comes in will use those mappings.

```auto
PUT _index_template/template_name
{
  "index_patterns": ["myindex-data-*"],
  "template": {
    "mappings": {
      "properties": {
        "TimeOfConnection": {
          "type": "date"
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 2, 2021, 4:55pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/27 "2021-03-02T16:55:16Z")

</div>

Im sorry I wasnt clear as there are some concepts in Elastic Stack that I am not yet understanding....

Is there a way to create dynamic created index but with some fields "force" them with a static type?

Currently, I have all my data incoming using a generic index template (I dont even know which honestly)

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 2, 2021, 5:44pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/28 "2021-03-02T17:44:20Z")

</div>

You probably don't have an index template you are currently using. When one doesn't exist it Elastic will guess the mapping.

If you insert the index template I put above and then clear out your old mapping/index then ingest the data again it should all work. You can add to the mapping if any other fields you are ingesting are of the incorrect type. Or you can leave as is and Elastic will map them for you.

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 3, 2021, 8:20am UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/29 "2021-03-03T08:20:24Z")

</div>

Im going to attempt to create one today and see if I can get it to work that way.

Illl try to post back updates.

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 3, 2021, 3:36pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/30 "2021-03-03T15:36:30Z")

</div>

OK so Ive attempted a few times to create a template index but seem to be failing.

In Kibana, I went to Stack Management, Data, Index Management and went to Index Templates. I create a template there, with the mapping like you mentioned. But in my Logstash conf file, I tried to make it use that template but it says it cannot be found.

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 3, 2021, 3:40pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/31 "2021-03-03T15:40:20Z")

</div>

Im trying to even search what Index Template is being applied to a index...

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 3, 2021, 3:44pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/32 "2021-03-03T15:44:20Z")

</div>

You can run `GET _index_template` to see all the templates. But if you never created one for this index then I doubt you will find it.

In Dev Tools run the below and it will create it for you. Change the `template_name` to something more unique.

```auto
PUT _index_template/template_name
{
  "index_patterns": ["myindex-data-*"],
  "template": {
    "mappings": {
      "properties": {
        "TimeOfConnection": {
          "type": "date"
        }
      }
    }
  }
}

```

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 3, 2021, 5:09pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/33 "2021-03-03T17:09:51Z")

</div>

So maybe:

```
PUT _index_template/myndex_template
{
  "index_patterns": ["myndex-*"],
  "template": {
    "mappings": {
      "properties": {
        "TimeOfConnection": {
          "type": "date"
        }
      }
    }
  }
}

```

And you are saying to paste it here?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/b/4bcfb445b55fb9ee4fe857d11021ed76c13b2a11.png)

This works without credentials?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 3, 2021, 5:14pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/34 "2021-03-03T17:14:59Z")

</div>

In Kibana there is Dev Tools

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/b/8b71d2c5d8bff1849937933c556a89dfb6c17d44.png)

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 3, 2021, 5:15pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/35 "2021-03-03T17:15:55Z")

</div>

I apoligize for my blindness and stupidity.

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 3, 2021, 5:18pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/36 "2021-03-03T17:18:15Z")

</div>

Well ACK true.....Let me see....

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 3, 2021, 5:20pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/37 "2021-03-03T17:20:00Z")

</div>

Still getting it as text....

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 3, 2021, 5:25pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/38 "2021-03-03T17:25:00Z")

</div>

Did you delete your index, index pattern, run the template API above, run logstash, create new index pattern? In that order.

Where are you seeing it as a string?

---

<div class="post-metadata">

**Author:** ![riahc3](https://avatars.discourse-cdn.com/v4/letter/r/d6d6ee/32.png) [@riahc3](https://discuss.elastic.co/u/riahc3)\
**Post date:** [March 4, 2021, 8:43am UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/39 "2021-03-04T08:43:24Z")

</div>

OK, so maybe Im doing something wrong. Lets confirm the steps.

1: Stop Logstash

systemctl stop logstash

2: Delete data index:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/7/4775ab077732c6990eee3cbe7248b454f1285694.png)

3: Delete Kibana Index Pattern:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/0/909fe9aba71355b5cb8bd9009c506460a28c2bab.png)

4: Run the template API:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bb01d26dca97543b683052f2b0c810e13552b757.png)

5: Start Logstash again:

systemctl start logstash

6: Thats it?

I do not have to change my Logstash conf?

---

<div class="post-metadata">

**Author:** ![aaron-nimocks](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aaron-nimocks/32/73965_2.png) [@aaron-nimocks](https://discuss.elastic.co/u/aaron-nimocks)\
**Post date:** [March 4, 2021, 1:12pm UTC](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594/40 "2021-03-04T13:12:31Z")

</div>

Step 4 has to be done in [Kibana Dev Tools Console](https://www.elastic.co/guide/en/kibana/current/console-kibana.html), not the browser developer console.

[Previous page](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594.md?page=1)

[Next page](https://discuss.elastic.co/t/convert-field-containing-a-timestamp-from-string-to-a-date-time-or-timestamp/265594.md?page=3)