Need Help with prefix search with highlighter

Elastic Stack Elasticsearch
1

Setting and mapping Looks:

{
 "settings" : {
  "analysis" : {
    "analyzer" : {
   "analyzer_startswith": {
    "tokenizer": "keyword",
    "filter": "lowercase"
   }
    }
  }
  "mappings" : {
   "part" : {
   "properties" : {
     "partNumber" : {
    "type" : "string",
    "search_analyzer" : "standard",
    "index_analyzer": "analyzer_startswith"
     },
     "crossReferencePartNumber" : {
    "type" : "string",
    "search_analyzer" : "standard",
    "index_analyzer": "analyzer_startswith"
     },
     "commercialDescription_de_DE" : {
    "type" : "string",
    "search_analyzer" : "analyzer_startswith",
    "index_analyzer": "standard"
     },
     "commercialDescription_en_US" : {
    "type" : "string",
     "search_analyzer" : "analyzer_startswith",
    "index_analyzer": "standard"
     }
   }
  }
 }
}

with a search query:

{
  "bool" : {
    "should" : {
      "match" : {
        "_all" : {
          "query" : "65",
          "type" : "phrase_prefix",
          "analyzer" : "standard",
          "max_expansions" : 10
        }
      }
    }
  }

following results are provided:

{
  "took" : 10,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 10,
    "max_score" : null,
    "hits" : [ {
      "_index" : "test",
      "_type" : "part",
      "_id" : "281508_1",
      "_score" : null,
      "_source":{"partNumber":"10161515","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X8 MM1000-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X8 MM1000-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "281508_5",
      "_score" : null,
      "_source":{"partNumber":"10161515","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X8 MM1000-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X8 MM1000-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "280496_5",
      "_score" : null,
      "_source":{"partNumber":"10161599","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X3X3390 T2+T3 BL5 MM800-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X3X3390 T2+T3 BL5 MM800-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "280496_1",
      "_score" : null,
      "_source":{"partNumber":"10161599","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X3X3390 T2+T3 BL5 MM800-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X3X3390 T2+T3 BL5 MM800-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "266681_5",
      "_score" : null,
      "_source":{"partNumber":"10161808","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X3.0 MM500-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X3.0 MM500-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "266681_1",
      "_score" : null,
      "_source":{"partNumber":"10161808","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X3.0 MM500-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X3.0 MM500-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "284623_1",
      "_score" : null,
      "_source":{"partNumber":"10173433","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X8.0 SPE10100-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X8.0 SPE10100-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "284623_5",
      "_score" : null,
      "_source":{"partNumber":"10173433","crossReferencePartNumber":null,"commercialDescription_en_US":"HYDR.-ROHR 65X8.0 SPE10100-6501","commercialDescription_de_DE":"HYDR.-ROHR 65X8.0 SPE10100-6501"}
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "410511_2",
      "_score" : null,
      "_source":{"partNumber":"6501","crossReferencePartNumber":["6501","LR501"],"commercialDescription_en_US":"3.990 dia locating ring Standard series","commercialDescription_de_DE":"3.990 dia locating ring Standard series"},
      "highlight" : {
        "partNumber" : [ "<b>6501</b>" ],
        "crossReferencePartNumber" : [ "<b>6501</b>" ]
      }
    }, {
      "_index" : "test",
      "_type" : "part",
      "_id" : "410527_2",
      "_score" : null,
      "_source":{"partNumber":"6501LN","crossReferencePartNumber":["6501 LN"],"commercialDescription_en_US":"3.990 dia locating ring LN series","commercialDescription_de_DE":"3.990 dia locating ring LN series"},
      "highlight" : {
        "partNumber" : [ "<b>6501LN</b>" ],
        "crossReferencePartNumber" : [ "<b>6501</b> LN" ]
      }
  }
 }
 }

what needs to be changed in the setting or query to get records which starts with 65 first.

when we do search with prefix

2

A couple things:

  1. I edited your post to wrap the code bits in ``` so it was readable.
  2. You are setting analyzers on your field but you are searching _all but you aren't changing its mapping. This is ok on the face of it but might be a mistake.
  3. I might analyze the two ways, once with keyword and lowercase filter and once with the standard analyzer and lowercase filter. Then you could search them both:
{
  "multi_match" : {
    "query" : "65",
    "fields": "partNumber^2, partNumber.keyword^10, crossReferencePartNumber, crossReferencePartNumber^5, other stuff you want in here"
    "type" : "phrase_prefix",
    "max_expansions" : 10
}

You have to use https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html to setup the .keyword bit on the fields.

3

Thanks Nik for a quick reply, we are trying with this option will update or ask for more help soon.

4

Nik,

Need your help again. We were able to solve one problem but we have much bigger problem.
For auto complete functionality this works perfectly, but when we want a results almost matching.
Say we have 2 fields defined as
Part Number: String
PartDescription: String

with values as
PartNumber: 6501
PartDescription: Guide

PartNumber: GD81
PartDescription: Guide 65

PartNumber: 6506
PartDescription: Guide Pin 65

PartNumber: 8014
PartDescription: ABCD

we would like to look at all the fields which matches 65 but results starting with 65 should come on top.

5

I think the thing to use in this case is the edge ngram tokenizer.