Normalizer lowercase not found

We develop an open source Java EE web application that uses Elasticsearch. This works wonderfully productively and very stable on countless servers. I now updated a single server, where it had been running smoothly so far, too. The index profile also changed, so I deleted all the indexes and wanted to recreate them. This is also standard for us during updates.

Now I get the following error message when creating the index profile:

IndexingForm - method [PUT], host [http://localhost:9200], URI [/kitodo_process], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"normalizer [lowercase] not found for field [keyword]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping [_doc]: normalizer [lowercase] not found for
 field [keyword]","caused_by":{"type":"mapper_parsing_exception","reason":"normalizer [lowercase] not found for field [keyword]"}},"status":400}

What could cause this? As I read elsewhere, the normalizer 'lowercase' is included in Elasticsearch, nothing extra needs to be installed here. But apparently it can't be found on this server. I also checked the elasticsearch.log but it didn't record anything about the error case.

OS Release : SLES 12 SP5

Elasticsearch version information:

{
  "name" : "sla901t",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "BqAMCaCYSnW3-gvQo94-AA",
  "version" : {
    "number" : "7.3.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "4749ba6",
    "build_date" : "2019-08-19T20:19:25.651794Z",
    "build_snapshot" : false,
    "lucene_version" : "8.1.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Java version:

/usr/share/elasticsearch/jdk/bin/java -version
openjdk version "12.0.2" 2019-07-16
OpenJDK Runtime Environment (build 12.0.2+10)
OpenJDK 64-Bit Server VM (build 12.0.2+10, mixed mode, sharing)

Index profile:

{
  "mappings": {
    "dynamic": "strict",
    "properties": {
      "id": {
        "type": "long"
      },
      "batches": {
        "properties": {
          "id": {
            "type": "long"
          },
          "title": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "type": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "creationDate": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss",
        "ignore_malformed": true
      },
      "docket": {
        "type": "long"
      },
      "processBaseUri": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "hasChildren": {
        "type": "boolean"
      },
      "inChoiceListShown": {
        "type": "boolean"
      },
      "baseType": {
        "type": "text",
        "fields": {
          "keyword": {
            "type":"keyword",
            "ignore_above": 256
          }
        }
      },
      "comments": {
        "properties": {
          "id": {
            "type": "long"
          },
          "message": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256,
                "normalizer": "lowercase"
              }
            }
          }
        }
      },
      "meta": {
        "dynamic": "false",
        "properties": {
          "ID": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "mdWrap": {
            "properties": {
              "MDTYPE": {
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword",
                    "ignore_above": 256
                  }
                }
              },
              "xmlData": {
                "properties": {
                  "kitodo": {
                    "properties": {
                      "kitodo": {
                        "type": "text",
                        "fields": {
                          "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                          }
                        }
                      },
                      "metadata": {
                        "type": "nested",
                        "properties": {
                          "name": {
                            "type": "text",
                            "fields": {
                              "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                              }
                            }
                          },
                          "content": {
                            "type": "text",
                            "fields": {
                              "keyword": {
                                "type": "keyword",
                                "ignore_above": 256
                              }
                            }
                          }
                        }
                      },
                      "metadataGroup": {
                        "properties": {
                          "metadata": {
                            "type": "nested",
                            "properties": {
                              "name": {
                                "type": "text",
                                "fields": {
                                  "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                  }
                                }
                              },
                              "content": {
                                "type": "text",
                                "fields": {
                                  "keyword": {
                                    "type": "keyword",
                                    "ignore_above": 256
                                  }
                                }
                              }
                            }
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      },
      "numberOfMetadata": {
        "type": "long"
      },
      "numberOfImages": {
        "type": "long"
      },
      "numberOfStructures": {
        "type": "long"
      },
      "project": {
        "properties": {
          "id": {
            "type": "long"
          },
          "active": {
            "type": "boolean"
          },
          "title": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256,
                "normalizer": "lowercase"
              }
            }
          },
          "client": {
            "properties": {
              "id": {
                "type": "long"
              }
            }
          }
        }
      },
      "parent": {
        "properties": {
          "id": {
            "type": "long"
          }
        }
      },
      "properties": {
        "dynamic": "false",
        "type": "nested",
        "properties": {
          "title": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "value": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }}
      },
      "ruleset": {
        "type": "long"
      },
      "sortHelperArticles": {
        "type": "long"
      },
      "sortHelperDocstructs": {
        "type": "long"
      },
      "sortHelperImages": {
        "type": "long"
      },
      "sortHelperMetadata": {
        "type": "long"
      },
      "sortHelperStatus": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        },
        "fielddata": true
      },
      "lastEditingUser": {
        "type": "keyword"
      },
      "processingBeginLastTask": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss",
        "ignore_malformed": true
      },
      "processingEndLastTask": {
        "type": "date",
        "format": "yyyy-MM-dd HH:mm:ss",
        "ignore_malformed": true
      },
      "correctionCommentStatus": {
        "type": "byte"
      },
      "progressClosed": {
        "type": "double"
      },
      "progressInProcessing": {
        "type": "double"
      },
      "progressOpen": {
        "type": "double"
      },
      "progressLocked": {
        "type": "double"
      },
      "progressCombined": {
        "type": "keyword"
      },
      "tasks": {
        "properties": {
          "id": {
            "type": "long"
          },
          "title": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "templates": {
        "properties": {
          "id": {
            "type": "long"
          }
        }
      },
      "workpieces": {
        "properties": {
          "id": {
            "type": "long"
          }
        }
      },
      "template": {
        "properties": {
          "id": {
            "type": "long"
          },
          "title": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      },
      "title": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256,
            "normalizer": "lowercase"
          },
          "raw": {
            "type": "text",
            "index": false
          }
        },
        "fielddata": true
      },
      "wikiField": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      }
    }
  }
}

Any suggestions, how I can get this machine up to working, will be appreciated!

Hi @M.Ronge

In settings, how do you configure the "normalizer"?
It would be something like this:

"settings": {
    "analysis": {
        "normalizer": {
            "lowercase": {
                "type": "custom",
                "filter": ["lowercase"]
            }
        }
    }
}

Adding your suggestion actually seems to fix that error. Thanks for the help! The snippet must be inserted at the top of the indexing profile, like:

{
  "settings": {
    "analysis": {
      "normalizer": {
        "lowercase": {
          "type": "custom",
          "filter": ["lowercase"]
        }
      }
    }
  },
  "mappings": {
    "dynamic": "strict",
    "properties": {
      ...

Open question is still: What is this one server different from the ≈50 other servers where the problem does not occur; what has changed here since the problem didn’t rise here before either (for years)? I don’t see the indexing part of the application was changed in the recent years, so I think it must be something with this instance, a version issue (but if I understand the version information correctly, that I reported in my first post, then it is a version 7, which is what the application should work with) or corrupted files or permissions or so.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.