# 'Intelligently' cutting out results

**URL:** https://discuss.elastic.co/t/intelligently-cutting-out-results/281194
**Category:** Elasticsearch
**Created:** [August 12, 2021, 11:20am UTC](https://discuss.elastic.co/t/intelligently-cutting-out-results/281194 "2021-08-12T11:20:48Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![slawosz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slawosz/32/56516_2.png) [@slawosz](https://discuss.elastic.co/u/slawosz)
#### Post date: [August 12, 2021, 11:20am UTC](https://discuss.elastic.co/t/intelligently-cutting-out-results/281194/1 "2021-08-12T11:20:48Z")

</div>

Hi,  
lets say I have products with serial numbers, and format is `FOO-XXXXX` where `FOO` is always there and XXXXX are digits.

When I search for `FOO-12345`, the results are showing all products, for example (score in brackets):

```auto
FOO-12345 (3)
FOO-12344 (2.5)
FOO-42353 (0.01)
FOO-XXXX (0.01)

```

Basically, all numbers where only `FOO` is being matched returns low score.

Soo basically, this data can be grouped into 2 distinct clusters score. Sadly, I don't know proper terminology, but one cluster is wider and close to 3, and second has very similar score close to 0.01. Is there a way to instruct elasticsearch, that in such case, return only first cluster?  
I am happy to do all the reading, as well learn/relearn required math, so all I am asking are good reads you can point me to.

Thanks

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 12, 2021, 1:41pm UTC](https://discuss.elastic.co/t/intelligently-cutting-out-results/281194/2 "2021-08-12T13:41:04Z")

</div>

Hi Sławosz

Scores are computed on a number of factors, some of which vary over time as more content is added to the index. For this reason we don't suggest reading too much into what the scores mean (i.e. a score of 1 doesn't mean "perfection").

That said, if you want consider the entire range of scores produced by a query and look at their distribution the `percentiles` aggregation can be used to help draw that curve:

```auto
GET /MY_INDEX/_search
{
  "query": {
     ... MY QUERY HERE ...
  },
  "aggs": {
    "scoreDistribution": {
      "percentiles": {
        "script": "_score"        
      }
    }
  }
}

```

The results in my test query here look like this:

```auto
  "aggregations" : {
    "scoreDist" : {
      "values" : {
        "1.0" : 5.383362350463867,
        "5.0" : 6.5666823387146,
        "25.0" : 7.337974548339844,
        "50.0" : 8.046831130981445,
        "75.0" : 9.65035629272461,
        "95.0" : 11.354881286621094,
        "99.0" : 14.382296962738014
      }
    }
  }

```

---

<div class="post-metadata">

### Author: ![slawosz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/slawosz/32/56516_2.png) [@slawosz](https://discuss.elastic.co/u/slawosz)
#### Post date: [August 12, 2021, 3:04pm UTC](https://discuss.elastic.co/t/intelligently-cutting-out-results/281194/3 "2021-08-12T15:04:54Z")

</div>

Thanks Mark,  
it is indeed very helpful. Its step in very good direction to potential solution.  
I got something like this:

```auto
"aggregations" : {
    "scoreDistribution" : {
      "values" : {
        "1.0" : 0.015267470851540565,
        "5.0" : 0.015267470851540565,
        "25.0" : 0.015267470851540565,
        "50.0" : 0.015267470851540565,
        "75.0" : 0.015267470851540565,
        "95.0" : 2.449463472701605,
        "99.0" : 3.1063098907470703
      }
    }
  }

```

As you can see, most of the results has poor score, and there is huge gap. Could you recommend a method how to detect this gap (I am not afraid of math)?

---

<div class="post-metadata">

### Author: ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)
#### Post date: [August 12, 2021, 3:27pm UTC](https://discuss.elastic.co/t/intelligently-cutting-out-results/281194/4 "2021-08-12T15:27:04Z")

</div>

> [@slawosz](#):
>
> Could you recommend a method how to detect this gap (I am not afraid of math)?

> [@slawosz](#):
>
> When I search for `FOO-12345` , the results are showing all products,

Perhaps a much simpler approach is to make all query terms required using the [AND operator.](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#match-field-params). In your query example that would be turned into a search for `FOO AND 12345` as opposed to the default `FOO OR 12345`.  
The details can depend on which query type and index mapping you are using.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 9, 2021, 3:27pm UTC](https://discuss.elastic.co/t/intelligently-cutting-out-results/281194/5 "2021-09-09T15:27:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
