Change in index template behavior?

Elastic Stack Elasticsearch
1

I recently upgraded from ES 2.2 to ES 2.4 (on the way to 5.x), and noticed an apparent change in behavior in how index templates are handled. Scanning the release notes didn't turn up anything that appeared to be related, so I a left to wonder whether this change was intentional, or some sort of regression.

The basic scenario, with configuration files shortened for readability.

First, I have my "common" template file, which sets some broad defaults, and defines a custom analyzer. The important bits look something like this:

{
  "template" : "*",
  "order"    : 0,
  "settings" : {
    "analysis" : {
      "analyzer" : {
        "my_custom_analyzer" : {
          "type"      : "custom",
          "tokenizer" : "whitespace",
          "filter"    : [
            ... filters here ...
          ]
        }
      },
      "filter" : {
        ... filter definitions here ...
      }
    }
  }
}

Next is the index specific template which takes care of my index specific mapping, and crucially, attempts to make use of an analyzer that was defined in the common template.

{
  "template" : "service-*",
  "order"    : 1,
  "settings" : {
    ... settings ...
  },
  "mappings" : {
    "_default_" : {
      "properties" : {
        "message" : {
          "type" : "string", 
          "index" : "analyzed",
          "analyzer" : "my_custom_analyzer"
        }
      }
    }
  }
}

This arrangement worked just fine in 2.2.x, with matching indices being created with all of the expected settings, and with the proper analyzer in place.

Once I upgraded to 2.4.x, however, I was unable to even PUT my second template (the service-* one) to the _template endpoint successfully. I receive this error on every attempt:

{
   "error": {
      "root_cause": [
         {
            "type": "mapper_parsing_exception",
            "reason": "analyzer [my_custom_analyzer] not found for field [message]"
         }
      ],
      "type": "mapper_parsing_exception",
      "reason": "Failed to parse mapping [_default_]: analyzer [my_custom_analyzer] not found for field [message]",
      "caused_by": {
         "type": "mapper_parsing_exception",
         "reason": "analyzer [my_custom_analyzer] not found for field [message]"
      }
   },
   "status": 400
}

While the samples above are not representative of the entire configuration, it is worth mentioning that I do not believe there are any syntax or structure errors in my 'real' configuration files. The same files were working on 2.2.x, but not on 2.4.x. Furthermore, removing the single reference to the custom analyzer allows everything to proceed normally, albeit without the analyzer I actually want.

My suspicion is that something change in the validation of the template file on PUT. While it seems entirely reasonable to validate that file for basic structural problems, this particular validation (e.g. is the analyzer defined in this template?) seems to be counter-productive, and will force me to redefine the analyzer across several different index templates.

Does anyone have any thoughts on whether this is intentional? a bug? something else?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.