Custom analyzer unable to find filter/tokenizers from other templates

Elastic Stack Elasticsearch
1

After upgrading to 2.4.0 our templates are no longer functioning from 2.3.3.

We have our template split up into several templates so we can specify default analyzers per language, and we share analyzers/filters/tokenizers between the templates.

Here is a simplified example of the problem:

POST /_template/analyzers
{
  "template": "*",
  "order": 20,
  "settings": {
    "analysis": {
      "analyzer": {
        "untouched_analyzer": {
          "type": "custom",
          "tokenizer": "keyword",
          "filter": [ "max_length", "lowercase" ]
        },
        "no_stopwords_analyzer": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [ "max_length", "standard", "lowercase" ]
        }
      },
      "tokenizer": {
        "standard": {
          "type": "standard",
          "version": "4.4"
        }
      },
      "filter": {
        "max_length": {
          "type": "length",
          "max": "32766"
        }
      }
    }
  }
}

POST /_template/analyzers-en
{
  "template": "*-en",
  "order": 30,
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "type": "custom",
          "tokenizer": "standard",
          "filter": [ "max_length", "standard", "lowercase", "stop_en" ]
        }
      },
      "filter": {
        "stop_en": {
          "type": "stop",
          "stopwords": "_english_",
          "ignore_case": "true"
        }
      }
    }
  }
}

This results in the following error:

2016-09-13 09:52:00,174][DEBUG][action.admin.indices.template.put] [Zartra] failed to put template [analyzers-en]
[kOccXLKbR5CwAsXCt0v_3w] IndexCreationException[failed to create index]; nested: IllegalArgumentException[Custom Analyzer [default] failed to find filter under name [max_length]];

I can provide the full stacktrace if necessary.

Interestingly, if it's not a custom analyzer, other analyzer seem to have no issues:

POST /_template/analyzers-cs
{
  "template": "*-cs",
  "order": 30,
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "type": "czech",
          "filter": [ "max_length" ]
        }
      }
    }
  }
}
2

Hi @mikeb7986
I replied https://github.com/elastic/elasticsearch/issues/20479#issuecomment-247063641 .
After 2.4, we validate template as a complete mapping and settings.
If you add max_length filter settings to each template, you get OK.

In your last example about default analyzer, elasticsearch ignores filter settings because czech ANALYZER does not support filter settings.

3

That makes sense, so it's not necessarily custom analyzers, we can continue the discussion on github.