# Index template with only one field across multiple indices

**URL:** <https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315>\
**Category:** Elasticsearch\
**Created:** [October 24, 2022, 1:37pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315 "2022-10-24T13:37:59Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 24, 2022, 1:37pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/1 "2022-10-24T13:37:59Z")

</div>

Hi,

I have multiple indices like mailbox-xx- xxxx .The xx-xxxx values are assigned dynamically. Each index contains various fields with date as a common field. The problem that date field has been read as long type by default , but I need to keep it as date type.

I tried to use \_index\_template API and specify only the date field in mappings, but I saw that data is not written to those indices since the mapping mismaches between the index template and actual query.

Is there anything can be done here?

Many thanks

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 24, 2022, 1:39pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/2 "2022-10-24T13:39:30Z")

</div>

What is the template you created? You need to share it.

Also, did you deleted the old indices and created new ones? Templates only work when creating the index.

---

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 24, 2022, 1:46pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/3 "2022-10-24T13:46:32Z")

</div>

Hi,

THank you for your reply.

Here is my template:

```auto
PUT _index_template/mailbox
{
   "index_patterns":[
      "mailbox*"
   ],
   "priority":"100",
   "template":{
      "mappings":{
         "properties":{
            "date":{
               "type":"date",
               "format":"epoch_second"
            }
         }
      }
   }
}

```

Now, after the template is created I start loading data to elasticsearch. The query has more fields than a date. The data is not written to the index.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 24, 2022, 1:57pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/4 "2022-10-24T13:57:03Z")

</div>

And what does your date string looks like?

Please share a sample of your log messages.

---

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 24, 2022, 2:01pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/5 "2022-10-24T14:01:52Z")

</div>

Hi,

Date is long type, I need to force it into date type.  
Below is the exmaple

```auto
 {
        "_index" : "mailbox-ps-2022.10",
        "_type" : "PS",
        "_id" : "A2C1418190071_227264124_1666616748",
        "_score" : 1.0,
        "_source" : {
          "emetteur" : "fxd600",
          "comment" : "defaut systeme",
          "reference" : "A2C1418190071",
          "SN" : "227264124",
          "date" : 1666616748,
          "site" : "trwwv"
        }
      }

```

Wil dynamic template within index template work for this?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 24, 2022, 2:07pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/6 "2022-10-24T14:07:19Z")

</div>

> [@not\_correct](#):
>
> Wil dynamic template within index template work for this?

No, your mapping is correct, it should work.

Did you deleted the old indices or created a new index?

How are you loading your data?

---

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 24, 2022, 2:11pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/7 "2022-10-24T14:11:03Z")

</div>

I'am on the version 7.16.  
I want to create mapping for one common filed across multiple indices.  
I deleted all the indices and created the mapping.  
I started loading the data. I use \_search api to see if the date field is mapped accordingly, but the indices are empty. If I remove the \_index\_template the loading works fine.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 24, 2022, 2:14pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/8 "2022-10-24T14:14:29Z")

</div>

I understand what you are trying to do, but you didn't explained **how** you are loading the data.

If after the template was applied you got no data in the index, this could mean some kind of mapping issue, and you will get an error while trying to index the data in elasticsearch.

What are you using to index the data in elasticsearch?

---

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 24, 2022, 2:34pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/9 "2022-10-24T14:34:25Z")

</div>

Thank you,

I see, thank you for your reply.  
I am not sure what client is used to load the data. It uses AWS Lambda python. The system is managed by another person so unfortunately I do not have much details.  
THe loading json looks like that:

```auto
{
  [
  {"index": {
        "_index": "mailbox-p2-2022.10",
        "_type": "P2",
        "_id": "AAA201834000003_826644600_1666615919"
    }
}
 \n
{
    "emetteur": "fxd9100n0002",
    "comment": "BiSac_lace",
    "reference": "AAA201834000003",
    "SN": "826644600",
    "operation": "092002",
    "face": "FACE2",
    "equipement": "000001",
    "programme": "E41B019FSB",
    "config": "Tog2",
    "nb_ci": "3",
    "date": 1666615919,
    "site": "fdsex",
    "Ligne": "smd0",
    "Designation": "Pc "
}
\n'
.......
]

```

Is there any other way to force a date type instead of default long?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 24, 2022, 2:45pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/10 "2022-10-24T14:45:03Z")

</div>

> [@not\_correct](#):
>
> Is there any other way to force a date type instead of default long?

No, the only way to force a data type to a field is using a index template.

Your index template is correct and it is expected to work, but if you are not getting any data you need to check in the source of the data, the response that the AWS Lambda is getting to see if there is any error.

You may also check in your Elasticsearch logs to see if there is any hint.

I suspect that this may be causing some issues:

```auto
"_type": "P2"

```

Types are deprecates in Elasticsearch 7.X and should not be used any more, but I do not know if this would cause an issue in your case.

---

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 25, 2022, 7:42am UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/11 "2022-10-25T07:42:50Z")

</div>

Hi,

Yes, you are right!  
I reviewed the reindexed indices, and saw that, when reindexed the original \_type is replaced by \_doc.

I need customized \_type for kibana visualization lib filters.  
At the same time some visualizations require date field in date format not long for average aggregation on date filed otherwise I am getting an error:

> Can't get text on a VALUE\_NULL at 1

Is there anyway, you think I can address this issue. The system in production and, I would eally like to avoid lots of changes?

Really appreciate your help, many thanks

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 25, 2022, 12:16pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/12 "2022-10-25T12:16:10Z")

</div>

> [@not\_correct](#):
>
> I reviewed the reindexed indices

So, you reindexed your indices after applying the template? You didn't shared this information before. The mapping is correct in the reindexed indices? If so, this means that the template is working.

> [@not\_correct](#):
>
> I need customized \_type for kibana visualization lib filters.

You should create a custom field for it or use the `tags` field, types are deprecated in version 7.X and removed in version 8.X you need to move away from it if you want to have less issues when upgrading.

> [@not\_correct](#):
>
> Is there anyway, you think I can address this issue. The system in production and, I would eally like to avoid lots of changes?

You may try to reindex it again, but this time specify the type destination index.

```auto
{
  "source": {
      "index": "sourceindex"
  },
  "dest": {
      "index": "destinationindex",
      "type": "destinationtype"
  }
}

```

Not sure if this will work, but it should according to the documentation for this version.

---

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 25, 2022, 1:37pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/13 "2022-10-25T13:37:51Z")

</div>

Thank you so much for your help.  
Now I see it is version compatibility issue.

> [@leandrojmp](#):
>
> So, you reindexed your indices after applying the template? You didn't shared this information before. The mapping is correct in the reindexed indices? If so, this means that the template is working.

I tried multiple options, one was to manually reindex the available indices. The problem with this approach: I do not know the index pattern that may appear. Yes , the mapping is correct, but the new data is not written because of the \_type field ( I have not yet tried your suggestions setting the \_type while reindexing)

I have probably the last question, at least at this stage. I reviewed the indices that have been restored from the snapshot. They have versions between 7.0 - 7.13.

If I decide to downgrade to 6.8, will they work ? or at least back to 7.0?  
The system is quite complex, was done by other person, I would like to skip rewriting all the queries, like replacing \_type, setting field.keyword /s

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 25, 2022, 1:51pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/14 "2022-10-25T13:51:41Z")

</div>

> [@not\_correct](#):
>
> If I decide to downgrade to 6.8, will they work ? or at least back to 7.0?

Elasticsearch does not support downgrade, you would need to install an entirely new cluster in a version that has reached EOL and will not receive more updates, you also cannot restore a snapshot in a cluster with a lower version than the cluster that created the snapshot.

I think that in your case, you may be able to temporarily solve your problem using a [runtime field](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/runtime.html), but be aware that this has an impact in performance.

A runtime field is a field mapping applied while reading, which means, while making queries, you may have your `date` field mapped as `long` in your index, but change it to a runtime field of the type date.

You will basically need to make the following request to your indices:

```auto
PUT index_name/
{
  "mappings": {
    "runtime": {
      "date": {
        "type": "date",
        "format": "epoch_second"
      }
    }
  }
}

```

And if you need to remove the runtime mapping for some reason, just use:

```auto
PUT index_name/
{
  "mappings": {
    "runtime": {
      "date": null
    }
  }
}

```

This may help your case, but it should not be the definitive solution, you need to change your system to not depend on the `_type` field.

---

<div class="post-metadata">

**Author:** ![not\_correct](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/not_correct/32/105273_2.png) [@not\_correct](https://discuss.elastic.co/u/not_correct)\
**Post date:** [October 26, 2022, 6:03pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/15 "2022-10-26T18:03:36Z")

</div>

Hi,  
I really appreciate your help, thanks a lot.  
I think I will try to remove the \_type field.

The one possible issue I may face is duplicating document \_id's. I understood specifying different \_type values allows having multiple documents with the same \_id.

I consider the option of setting different indices with patterns like : initialIndex\_initialType\_month and setting a custom tag so I can use filters on index pattern like initialIndex\* .

Anyway, thank you so much for your help, I helped a lot, without that I would definitely be blocked without seeing light at the end of the tonnel.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 23, 2022, 6:04pm UTC](https://discuss.elastic.co/t/index-template-with-only-one-field-across-multiple-indices/317315/16 "2022-11-23T18:04:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.