Match_phrase_prefix on keyword field inconsistent behavior between ES 5.6.8 and ES 6+

I'm noticing different behavior on Elasticsearch 5.6.8 and Elasticsearch 6+ on doing a match_phrase_prefix on keyword. I was able to do a match_phrase_prefix on a keyword exact match on Elasticsearch 5.6.8 but not on Elasticsearch 6.

I suspect it's because of the lowercase normalizer not being applied upon query in ES 6. Issuing a match_phrase_prefix in lowercase returns the document, but doesn't return anything if issued an uppercase

Step to reproduce:

// Create a an index
curl -X PUT 'localhost:9200/my_index_6'

// Create index mapping
    curl -X PUT 'localhost:9200/my_index_6/records/_mapping?pretty' -H 'Content-Type: application/json' -d '
    {
        "records": {
            "dynamic": "strict",
            "properties": {
                "my_field": {
                    "type": "keyword",
                    "fields": {
                        "caseinsensitive": {
                            "type": "keyword",
                            "ignore_above": 10922,
                            "normalizer": "lowercaseanalyzer"
                        }
                    }
                }
            }
        }
    }'

 // Add settings
    curl -X PUT 'localhost:9200/my_index_6' -H 'Content-Type: application/json' -d '
    {
        "settings": {
            "index": {
              "analysis": {
                "filter": {
                  "english_stopwords": {
                    "type": "stop",
                    "stopwords_path": "stopwords/stopwords_all.txt"
                  },
                  "english_stemmer": {
                    "name": "light_english",
                    "type": "stemmer"
                  },
                  "english_possessive_stemmer": {
                    "name": "possessive_english",
                    "type": "stemmer"
                  },
                  "english_stemmer_override": {
                    "type": "stemmer_override",
                    "rules_path": "stemmer_override/stemmer_override_english.txt"
                  }
                },
                "normalizer": {
                  "lowercaseanalyzer": {
                    "filter": [
                      "lowercase"
                    ],
                    "type": "custom"
                  }
                },
                "analyzer": {
                  "english": {
                    "filter": [
                      "english_possessive_stemmer",
                      "lowercase",
                      "english_stopwords",
                      "english_stemmer_override",
                      "english_stemmer"
                    ],
                    "type": "custom",
                    "tokenizer": "standard"
                  }
                }
              }
            }
          }    
    }
    '

// Create a document
curl -X POST 'localhost:9200/my_index_6/records' -H 'Content-Type: application/json' -d '
{
    "my_field": "ID01"
}
'

// Issue a search query. This WORKS on ES 5 but DOESN'T WORK on ES 6
curl -X GET "localhost:9200/my_index_6/_search?pretty" -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_phrase_prefix" : {
            "my_field.caseinsensitive" : {
                "query" : "ID01",
                "boost" : 1.0
            }
        }
    }
}
'

// THIS WORKS 
    curl -X GET "localhost:9200/my_index_6/_search?pretty" -H 'Content-Type: application/json' -d'
    {
        "query": {
            "match_phrase_prefix" : {
                "my_field.caseinsensitive" : {
                    "query" : "id01",
                    "boost" : 1.0
                }
            }
        }
    }
    '

Appreciate any help on this.

Thanks

@warkolm thoughts on this? In case you are not the right person, can you please redirect this to someone who can help?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.