# Partial match with ngram. How to avoid\\minimize excessive search result

**URL:** <https://discuss.elastic.co/t/partial-match-with-ngram-how-to-avoid-minimize-excessive-search-result/126785>\
**Category:** Elasticsearch\
**Created:** [April 4, 2018, 4:31pm UTC](https://discuss.elastic.co/t/partial-match-with-ngram-how-to-avoid-minimize-excessive-search-result/126785 "2018-04-04T16:31:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Denis\_Zakharchuk](https://avatars.discourse-cdn.com/v4/letter/d/cc9497/32.png) [@Denis\_Zakharchuk](https://discuss.elastic.co/u/Denis_Zakharchuk)\
**Post date:** [April 4, 2018, 4:31pm UTC](https://discuss.elastic.co/t/partial-match-with-ngram-how-to-avoid-minimize-excessive-search-result/126785/1 "2018-04-04T16:31:33Z")

</div>

Hi,  
I have a task to implement search among some records by some different fields.  
One of them - "Title" and regarding this specific field there are few demands:

- internal match (e.g. "esti" in search request should match "Testing" value);
- not strict match (e.g. "Tosting" in search request should match "Testing" value);

Another demand regarding search overall is that by default search output have to be displayed in chronological order (the later documents is - the better, there is a "CreationTime" field in document)

I have implemented first part (both internal and non-strict matches) using ngram token filter.  
Second part (chronological order), implemented by using "sort" feature in search query.  
My problem is that search on ngrammed field outputs to many non-relevant results  
For example if request was "Computer" then output contains documents with Titles

- "Computer"
- "Compressor"
- "Company"

Those non-relevant results, ofcourse, have lower \_score then documents with "Computer" in Title. But sorting by CreationTime neutralizes that so the users often see non-relevant results at the top (because they was created later).

My overall question - how can I work around this situation?

Few ways I can tell right away:

- Reimplement search technique to make it more "specific" (some how minimize redundancy of search output in the first place), so I can sort those results;
- Use some kind of factorisation (using FunctionScore compound query and use some factor based on CreatedDate (e.g. days from 2012-01-01 to CreationTime) as field value factor);

I have totally failed in both directions, so I'm asking you, guys, to help me.

Now, being more specific:  
Elastic Version - 6.2.2  
Index Setting\Mapping:

```
    {
   "tendersearch":{
      "aliases":{

      },
      "mappings":{
         "_doc":{
            "properties":{
               "@timestamp":{
                  "type":"keyword"
               },
               "@version":{
                  "type":"keyword"
               },
               "DateModified":{
                  "type":"date"
               },
               "Id":{
                  "type":"keyword"
               },
               "tender":{
                  "properties":{
                     
                     "CreationTime":{
                        "type":"date"
                     },
                    
                     "Factor":{
                        "type":"long"
                     },
                     "Title":{
                        "type":"text",
                        "fields":{
                           "ngram":{
                              "type":"text",
                              "analyzer":"trigrams"
                           },
                           "raw":{
                              "type":"keyword"
                           }
                        },
                        "analyzer":"word_delim_analyzer"
                     }                     
                  }
               }
            }
         }
      },
      "settings":{
         "index":{
            "number_of_shards":"1",
            "provided_name":"tendersearch",
            "max_result_window":"2147483647",
            "creation_date":"1522832508985",
            "analysis":{
               "filter":{
                  "trigrams_filter":{
                     "type":"ngram",
                     "min_gram":"4",
                     "max_gram":"4"
                  },
                  "word_delim_catenate":{
                     "catenate_all":"true",
                     "type":"word_delimiter"
                  }
               },
               "analyzer":{
                  "trigrams":{
                     "filter":[
                        "lowercase",
                        "word_delim_catenate",
                        "trigrams_filter"
                     ],
                     "type":"custom",
                     "tokenizer":"whitespace"
                  },
                  "word_delim_analyzer":{
                     "filter":[
                        "lowercase",
                        "word_delim_catenate"
                     ],
                     "type":"custom",
                     "tokenizer":"whitespace"
                  }
               }
            },
            "number_of_replicas":"1",
            "uuid":"kY7S9_NhSyCgWi047v0rSA",
            "version":{
               "created":"6020199"
            }
         }
      }
   }
}

```

Search Request Sample:  
{  
"size":10,  
"from":0,  
"sort":[  
{  
"tender.CreationTime":{  
"order":"desc"  
}  
},  
"\_score"  
],  
"query":{  
"bool":{  
"must":[  
{  
"match":{  
"tender.Title.ngram":{  
"query":"Computer"  
}  
}  
}  
]  
}  
}  
}

Any help is hightly appreciated!

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [April 10, 2018, 9:21am UTC](https://discuss.elastic.co/t/partial-match-with-ngram-how-to-avoid-minimize-excessive-search-result/126785/2 "2018-04-10T09:21:53Z")

</div>

I'd go with a function\_score query. Something like this would work:

```auto
{
  "size": 10,
  "from": 0,
  "query": {
    "function_score": {
      "query": {
        "match": {
          "tender.Title.ngram": {
            "query": "Computer"
          }
        }
      },
      "functions": [
        {
          "gauss": {
            "tender.CreationTime": {
              "origin": "now",
              "scale": "7d",
              "offset": "5d",
              "decay": 0.5
            }
          }
        }
      ]
    }
  }
}

```

This query will calculate a number between 0 and 1 based on how long ago the CreationTime is (based on a Gaussian function) and then multiply that number with the score of the match on the ngrammed Title field.

You can play with the [parameters like `scale` and `offset`](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#function-decay) a bit to let the gauss function go to 0 faster or slower.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 8, 2018, 9:21am UTC](https://discuss.elastic.co/t/partial-match-with-ngram-how-to-avoid-minimize-excessive-search-result/126785/3 "2018-05-08T09:21:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.