Convert field containing a timestamp from string to a date / time or timestamp?

Elastic Stack Logstash
41

Oh yeah sorry....Used a old screenshot

42

I tried this:

But it errors:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "x_content_parse_exception",
        "reason" : "[5:3] [index_template] unknown field [mappings]"
      }
    ],
    "type" : "x_content_parse_exception",
    "reason" : "[5:3] [index_template] unknown field [mappings]"
  },
  "status" : 400
}

Thought that would have done the trick

43

Your structure looks wrong. Do a GET _index_template/ and you can see all the other templates and use the same structure they use.

If you are still having issues I would open a new topic with that specific error you are encountering.

44

Yeah I see the structure but the issue is that I dont know which is the DEFAULT one that is being applied ..... Elastic Stack does not make that clear for some reason.

These are all the index templates I have after doing a GET _index_template/

In there, there should be one related to logstash correct?

45

OK, I sorta figured it out

0: Stop logstash
1: Delete index data
2: Delete index pattern
3: Start logstash
4: Generate data
5: Generate index pattern
6: Open Dev Tools and

PUT _index_template/your-index-name-template
{
  "index_patterns": ["your-index-name--*"],
  "template": {
    "mappings": {
      "properties": {
        "OneFieldWithDate": {
          "type": "date"
        },
		 "AnotherFieldWithDate": {
          "type": "date"
        }
      }
    }
  }
}

6: See data added
7: Stop logstash
8: Delete Index data
9: Delete Kibana data
10: Open Dev Tools and

PUT _index_template/your-index-name-template
{
  "index_patterns": ["your-index-name--*"],
  "template": {
    "mappings": {
      "properties": {
        "OneFieldWithDate": {
          "type": "date"
        },
		 "AnotherFieldWithDate": {
          "type": "date"
        }
      }
    }
  }
}

11: Start logstash
12: Generate data
13: Create Kibana index

Now I am getting the field in date....

Way way way too weird

1 Like
46

I have been able to cut it down to less steps :slight_smile:

Thats being said, I want to explore if there are other ways to do it: More specifically without deleting older data.

Ive thought about SQL where you can technically do a alter table and that allows you to change the column type.

Does anything like this exist for Elasticsearch indexes?

47

Once a field type is set in elasticsearch it cannot be changed. You would need to create a new index, apply a template that sets the field types you want, then copy the data from the old index to the new. The reindex API might be useful for that.

48

My indexes are created daily (for example: adlogins-2021-03-09, adlogins-2021-03-10, etc). Is there a way to do it this way?

49

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.