Elasticsearch 8.1 normalizer in strings error

Elastic Stack Elasticsearch
1

Hello, I've been using the next component template in elastic 7. I recently upgraded to elastic 8.1 and it won't let me create the template like before.

PUT _component_template/keywords
{
  "template": {
    "mappings": {
      "settings": {
        "analysis": {
          "normalizer": {
            "lowerascii": {
              "type": "custom",
              "char_filter": [],
              "filter": [
                "lowercase",
                "asciifolding"
              ]
            }
          }
        }
      },
      "dynamic_templates": [
       {
          "keywords": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword",
              "normalizer": "lowerascii"
            }
          }
        }
      ]
    }
  },
  "version": 2,
  "_meta": {
    "description": "All strings are keywords"
  }
}

and the following error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Failed to parse mapping: dynamic template [keywords] has invalid content [{\"match_mapping_type\":\"string\",\"mapping\":{\"normalizer\":\"lowerascii\",\"type\":\"keyword\"}}], attempted to validate it with the following match_mapping_type: [string]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping: dynamic template [keywords] has invalid content [{\"match_mapping_type\":\"string\",\"mapping\":{\"normalizer\":\"lowerascii\",\"type\":\"keyword\"}}], attempted to validate it with the following match_mapping_type: [string]",
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "dynamic template [keywords] has invalid content [{\"match_mapping_type\":\"string\",\"mapping\":{\"normalizer\":\"lowerascii\",\"type\":\"keyword\"}}], attempted to validate it with the following match_mapping_type: [string]",
      "caused_by" : {
        "type" : "mapper_parsing_exception",
        "reason" : "normalizer [lowerascii] not found for field [__dynamic__keywords]"
      }
    }
  },
  "status" : 400
}
2

Settings should be outside of Mapping.
This is a basic rule, not changed between es7 & es8

PUT _component_template/keywords
{
  "template": {
    "settings": {
      "analysis": {
        "normalizer": {
          "lowerascii": {
            "type": "custom",
            "char_filter": [],
            "filter": [
              "lowercase",
              "asciifolding"
            ]
          }
        }
      }
    },
    "mappings": {
      "dynamic_templates": [
        {
          "keywords": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword",
              "normalizer": "lowerascii"
            }
          }
        }
      ]
    }
  },
  "version": 2,
  "_meta": {
    "description": "All strings are keywords"
  }
}
1 Like
3

Hello Yassine!, in fact I have two diferent component templates. I wrote this for simplicity.
I have:

GET /_component_template/mc_normalizer
{
  "component_templates" : [
    {
      "name" : "mc_normalizer",
      "component_template" : {
        "template" : {
          "settings" : {
            "index" : {
              "analysis" : {
                "normalizer" : {
                  "lower_ascii_normalizer" : {
                    "filter" : [
                      "lowercase",
                      "asciifolding"
                    ],
                    "type" : "custom",
                    "char_filter" : [ ]
                  }
                }
              }
            }
          }
        },
        "version" : 2,
        "_meta" : {
          "description" : "Lower ascii"
        }
      }
    }
  ]
}

and I'm trying to add:

PUT _component_template/keywords
{
  "template": {
    "mappings": {
      "dynamic_templates": [
       {
          "keywords": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword",
              "normalizer": "lower_ascii_normalizer"
            }
          }
        }
      ]
    }
  },
  "version": 2,
  "_meta": {
    "description": "All strings are keywords"
  }
}

And the error is the same above. Thanks a lot!

4

I still don't see the issue with your logic if your run both in order

PUT /_component_template/normalizer
{
  "template": {
    "settings": {
      "index": {
        "analysis": {
          "normalizer": {
            "lower_ascii_normalizer": {
              "filter": [
                "lowercase",
                "asciifolding"
              ],
              "type": "custom",
              "char_filter": []
            }
          }
        }
      }
    }
  },
  "version": 2,
  "_meta": {
    "description": "Lower ascii"
  }

}

PUT _component_template/keywords
{
  "template": {
    "mappings": {
      "dynamic_templates": [
        {
          "keywords": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword",
              "normalizer": "lower_ascii_normalizer"
            }
          }
        }
      ]
    }
  },
  "version": 2,
  "_meta": {
    "description": "All strings are keywords"
  }
}
5

I've launched again in kibana, and the same error:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "Failed to parse mapping: dynamic template [keywords] has invalid content [{\"match_mapping_type\":\"string\",\"mapping\":{\"normalizer\":\"lower_ascii_normalizer\",\"type\":\"keyword\"}}], attempted to validate it with the following match_mapping_type: [string]"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "Failed to parse mapping: dynamic template [keywords] has invalid content [{\"match_mapping_type\":\"string\",\"mapping\":{\"normalizer\":\"lower_ascii_normalizer\",\"type\":\"keyword\"}}], attempted to validate it with the following match_mapping_type: [string]",
    "caused_by" : {
      "type" : "illegal_argument_exception",
      "reason" : "dynamic template [keywords] has invalid content [{\"match_mapping_type\":\"string\",\"mapping\":{\"normalizer\":\"lower_ascii_normalizer\",\"type\":\"keyword\"}}], attempted to validate it with the following match_mapping_type: [string]",
      "caused_by" : {
        "type" : "mapper_parsing_exception",
        "reason" : "normalizer [lower_ascii_normalizer] not found for field [__dynamic__keywords]"
      }
    }
  },
  "status" : 400
}
6

Yes, I see your issue now, use the second part (dynamic mapping) in the _index_template

PUT _index_template/keywords
{
  "index_patterns": [
    "index-name-*"
  ],
  "template": {
    "mappings": {
      "dynamic_templates": [
        {
          "keywords": {
            "match_mapping_type": "string",
            "mapping": {
              "type": "keyword",
              "normalizer": "lower_ascii_normalizer"
            }
          }
        }
      ]
    }
  },
  "version": 2,
  "composed_of": [
    "normalizer"
  ],
  "_meta": {
    "description": "All strings are keywords"
  }
}
7

Thanks a lot. Now is working. I don't understand why is working fine in 7.16.2 but not un 8.1.0.
I have it in a python script and I've launch in both elastics.
I've change it, and its working!

8

Didn't figured out what exactly changed
I will check and share if i found any trace for a breaking change may be

1 Like
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.