ES default date type not taking nanos

[samee_alam]

ES version 7.17.3

Fluentd config is as below:

    time_key timestamp
    time_format %Y-%m-%dT%H:%M:%S.%N%Z

which takes nano seconds into consideration and data is stored properly in elasticsearch as:

But default data type in mapping properties of index is "date"

This is creating problem in sorting, as type "date" considers only milli seconds and documents are being sorted in wrong order for whom the difference is at nano second level.

How do I set the default date type as "date_nanos" ?

I know I can delete and re-create the index and type date_nanos and reindex it, but I cannot do this in production. So basically, by default it should take date_nanos.

Please suggest how I can configure this.
Also I am using helm charts, so configuration to be done at ES chart possible?

[leandrojmp (Leandro Pereira)]

@samee_alam

I know I can delete and re-create the index and type date_nanos and reindex it, but I cannot do this in production. So basically, by default it should take date_nanos.

Unfortunately i think this is the only way to change, you need to explictily map the field to use date_nanos, so for any existing date you need to change the mapping and reindex.

For new indices you can use a dynamic template to map any date type as date_nanos.

Something like this, I guess:

    "dynamic_templates": [
      {
        "date_nano": {
          "match_mapping_type": "date",
          "mapping": {
            "type": "date_nanos"
          }
        }
      }

[samee_alam]

Thanks for your response.
As suggested I am creating an index template and mapped the field "date" to "date_nanos".

Once ES is up, as part of the readiness probe, using curl, I am pushing this to ES, so once data starts coming to ES, date is taken as date_nanos.