Use an analyzer and a normalizer at the same time on the same field?

Elastic Stack Elasticsearch
1

Hi,

for some reason, if need to query (prefix) with an insensitive case on a text field, but I would like to keep the advantage of the tokenisation made by the default Analyzer on the same field, is it possible ?

Example
My indexed field (let's say three cases from input)
"productSearchName" : "best white chocolate"
"productSearchName" : "Best White Chocolate"
"productSearchName" : "BEST WhItE ChocolatE"

If I apply the standard Analyzer, the produced tokens will be
["best","white","chocolate"] for three syntax, but how to perform a Prefix query (start with "best white") ? Using the productSearchName.Keyword is impossible since the sensitive case.
However I can perform match query on any part of the indexed tokens.

If I apply a Normalizer like this one

      "analysis": {
        "normalizer": {
          "case_insensitive": {
            "filter": [
              "lowercase"
            ],
            "type": "custom"
          }
        }

then for the same productSearchName input, I 'll get only one token [ "best white chocolate"] which'll allow me to perform case insensitive prefix query, but now I can't easily perform a match query, I need to use a wildcard instead.

Well can someone tell me if both an Analyser and a Normalizer can be used by the same field producing at the same time the tokens ["best","white","chocolate"] and [ "best white chocolate"] allowing me to keep the power of match query (no wildcard then) and perform case insensitive prefix query at the same time ?

Many thanks

2

Side note: A normalizer only works on keyword fields, so I do not think that this is an option here.

First, you may want to take a look at multi fields allowing you to index a single field in different ways in Elasticsearch.

And, just as another point to read about: There is a dedicated search as you type field type that could help your use-case.

Hope that helps to get started!

3

@spinscale

thanks for your quick reply, well I use C# Nest lib with the AutoMap() method which give by default for all "string" fields a "text" type and a keyword.

    "stringTypeField": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    }

Is it possible to have such mapping for a field (standard Analyzer for text and normalizer for keyword) ?

    "productType": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "normalizer": "case_insensitive"
        }
      }
    }

I'll have a look at search_as_you_type which may as you say solve my problem!

4

@spinscale
The search_as_you_type mapping is awesome!! Combining best of two worlds.
and @abdon told me about v7.10 which'll allows to perform prefix queries on Keyword with a case insensitive parameter (no more Normalizer, "just" for that)

Both of you made my day, thank you very much guys :slight_smile:

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.