Raise Elasticsearch field limit via config

[daniel90 (Daniel)]

I tried the obvious, but that gives me the:
"node settings must not contain any index level settings"

Error

So how do i do it?

I want a permanent solution that does not involve a cronjob setting the limit to X every 6 hours to account for potential restarts.

Now that I think about it that is not THAT bad. But just putting it in the config would be way cleaner.

[Musab_Dogan]

You can use index template to automate this process. Beats and other elasticsearch components using that way.

PUT _index_template/index-template-for-field-limit
{
  "template": {
    "settings": {
      "index.mapping.total_fields.limit": "10000"
    }
  },
  "index_patterns": [
    "index-name-*"
  ]
}

References:

Mapping limit settings | Elasticsearch Guide [8.15] | Elastic

Index templates | Elasticsearch Guide [8.15] | Elastic

[Wolfram_Haussig]

Hello Daniel,

I recommend to use Index Templates for this. It allows to set this setting for all indices that match the index template pattern.
You could even go so far to extract the setting to a component template and reuse it in multiple Index Templates.

I would advise to be cautious with this setting and only enabling it where absolutely necessary as this limit is there for a reason:

The limit is in place to prevent mappings and searches from becoming too large. Higher values can lead to performance degradations and memory issues, especially in clusters with a high load or few resources.

If you increase this setting, we recommend you also increase the indices.query.bool.max_clause_count setting, which limits the maximum number of clauses in a query.

Best regards
Wolfram