Template setting is not honored by the index

Elastic Stack Elasticsearch
1

I created an index template with

settings: {
        'index.mapping.total_fields.limit': 1500,
        'index.mapping.ignore_malformed': true
      }

After that I create an index with matching pattern. But the index hits the 1000 limit. If I put the setting directly on the index it works. And in the Kibana I can see the setting is set on the template.

I am on version 7.

2

Exactly which Elasticsearch version are you using?

I tried the following on 7.17 and it worked just fine:

curl -X PUT "localhost:9200/_index_template/testtemplate?pretty" -H 'Content-Type: application/json' -d'
{
  "index_patterns": ["test*"],
  "template": {
    "settings": {
      "index.mapping.total_fields.limit": 1500,
      "index.mapping.ignore_malformed": true
    }
  },
  "priority": 500
}
'

curl -X PUT "localhost:9200/test-index"

curl -X GET "localhost:9200/test-index/_settings?pretty"
{
  "test-index" : {
    "settings" : {
      "index" : {
        "routing" : {
          "allocation" : {
            "include" : {
              "_tier_preference" : "data_content"
            }
          }
        },
        "mapping" : {
          "total_fields" : {
            "limit" : "1500"
          },
          "ignore_malformed" : "true"
        },
        "number_of_shards" : "1",
        "provided_name" : "test-index",
        "creation_date" : "1719146898559",
        "number_of_replicas" : "1",
        "uuid" : "Ktywh9KrSU2rWu-DoJotQA",
        "version" : {
          "created" : "7171899"
        }
      }
    }
  }
}
3

Guess you missed the index GET response.

4

Correct. Edited the post and added it.

What did I do different from what you did?

5

I don't interact with ES directly. I am using a JS library to create templates and indices. I can see in Kibana that the setting is set in the template. I also see that Kibana shows my template as legacy.

One other thing is that I didn't set priority.

My version is 7.17 by the way.

6

If you are using legacy templates the format is different and you send it to a different endpoint. One of the differences is that priority is replaced by order.

7

There is an order field which I did not fill. I thought it is used when there are other templates but to my knowledge we have only the one I made with a matching pattern. Now should I set it like to 100 or something?

8

Yes, that is the first thing I would try. Once it has been uploaded I would then list all templates and see if there are any other templates that could apply to the index being created and adjust the order so the new template takes effect.

9

No use. I set order to 500 and PUT the template. Made sure the order is set in Kibana. Then created the index with a new suffix. But index settings in Kibana shows the setting is still set to 1000. And there are no other template matching the index name.

10

Please show the full template here (as it exists in Elasticsearch) so the issue can be recreated. As you are using the legacy format you also need to check if there are any new style index templates that match, as new style templates take precedence over legacy templates.

11

There is a new style template there. I placed the setting in it and recreated the index. The setting had been set [were observed] in the new index. So we can conclude that the problem was that template. After that I checked the index mapping and saw it did not take it from the template. I guess we can blame that template too. But that template has no mapping. What I expected was that the mapping be taken from my legacy template (as the new style one did not contain any) but this is not the case.