# Boosting a field yields bizarre results

**URL:** <https://discuss.elastic.co/t/boosting-a-field-yields-bizarre-results/46767>\
**Category:** Elasticsearch\
**Created:** [April 8, 2016, 6:44am UTC](https://discuss.elastic.co/t/boosting-a-field-yields-bizarre-results/46767 "2016-04-08T06:44:30Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Pyppe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pyppe/32/3531_2.png) [@Pyppe](https://discuss.elastic.co/u/Pyppe)\
**Post date:** [April 8, 2016, 6:44am UTC](https://discuss.elastic.co/t/boosting-a-field-yields-bizarre-results/46767/1 "2016-04-08T06:44:30Z")

</div>

Howdy!

I'm trying to boost search results by a status field (it's an enumeration). My query is as follows:

```auto
{
  "query" : {
    "bool" : {
      "must" : ...,
      "should" : [
        {
          "term" : {
            "status" : { "value" : "Valid", "boost" : 5 }
          }
        },
        {
          "term" : {
            "status" : { "value" : "Pending", "boost" : 4 }
          }
        },
        {
          "term" : {
            "status" : { "value" : "Expired", "boost" : 3 }
          }
        },
        {
          "term" : {
            "status" : { "value" : "Invalid", "boost" : 2 }
          }
        },
        {
          "term" : {
            "status" : { "value" : "Unknown", "boost" : 1 }
          }
        }
      ]
    }
  }
}

```

However, this does not sort the results as I would expect. I get the results as follows:

```auto
{
  "hits" : [
    "max_score": 2.1227207,
    "hits": [
      {
        "_score": 2.1227207,
        "_source": { "status": "Pending" }
      },
      {
        "_score": 1.5702559,
        "_source": { "status": "Unknown" }
      },
      {
        "_score": 1.4841971,
        "_source": { "status": "Valid" }
      },
      {
        "_score": 1.4767004,
        "_source": { "status": "Expired" }
      }
    ]
  ]
}

```

Obviously I was aiming to have Valid results first, then Pending, ..., and last Unknown.

If I remove the should boolean-array of boosted statuses altogether, then all the results have the same score (as I would suspect).

Am I missing something? Why is this query not working as I would suspect? [https://www.elastic.co/guide/en/elasticsearch/guide/current/\_boosting\_query\_clauses.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/_boosting_query_clauses.html) would IMO suggest that this is something that would be possible...

Thank you!

EDIT:

- the `status`-field mapping is `not_analyzed` that's why I use case-sensitive statuses in the query

---

<div class="post-metadata">

**Author:** ![Pyppe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pyppe/32/3531_2.png) [@Pyppe](https://discuss.elastic.co/u/Pyppe)\
**Post date:** [April 8, 2016, 7:30am UTC](https://discuss.elastic.co/t/boosting-a-field-yields-bizarre-results/46767/2 "2016-04-08T07:30:28Z")

</div>

Someone in the elasticsearch IRC-channel suggested to use the explain API. These are the results, if someone can make some sense out of them.

So this the explain-result for a document with `status: "Valid"` (which I assume would be sorted as first):

- [Explain for Valid document](https://gist.github.com/Pyppe/39d75137559dd1495c686cb0f6811878#file-explain_for_valid_document-json)

And this the `Pending`-result which gets a higher score than `Valid`, eventhough Valid should have a higher boost:

- [Explain for Pending document](https://gist.github.com/Pyppe/39d75137559dd1495c686cb0f6811878#file-explain_for_pending_document-json)

---

<div class="post-metadata">

**Author:** ![Pyppe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pyppe/32/3531_2.png) [@Pyppe](https://discuss.elastic.co/u/Pyppe)\
**Post date:** [April 8, 2016, 8:08am UTC](https://discuss.elastic.co/t/boosting-a-field-yields-bizarre-results/46767/3 "2016-04-08T08:08:49Z")

</div>

Ok... So I read [https://www.elastic.co/guide/en/elasticsearch/guide/current/relevance-intro.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/relevance-intro.html) and I guess the IDF-factor is to blame here.

If I adjust the boost values further from each other, then it will work. Feels kludgy, though... Any better options out there?

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [April 8, 2016, 1:55pm UTC](https://discuss.elastic.co/t/boosting-a-field-yields-bizarre-results/46767/4 "2016-04-08T13:55:31Z")

</div>

Yeah, this can be unintuitive at first. Like you discovered, the IDF can sometimes mess with your scoring, because it is trying to weight the relative rarity of the terms.

If you just don't care about the IDF, the easiest approach is to wrap your `term` queries in `constant_score` queries. That will assign a score of 1 if the doc matches, 0 otherwise. The TF/IDF are not taken into account. You can also boost them relative to each other for proper ordering. See this for more details: [https://www.elastic.co/guide/en/elasticsearch/guide/current/ignoring-tfidf.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/ignoring-tfidf.html)

You could also use the `function_score`to setup an array of filters which are each weighted differently. The advantage to `function_score` is that it is a lot more flexible (you can include many different sub-clauses, mathematical functions, etc) and can apply non-normalized boosts. The downside is that it's a lot more complicated 🙂

More details about `function_score` here: [https://www.elastic.co/guide/en/elasticsearch/guide/current/function-score-filters.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/function-score-filters.html) and here: [https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html)

_(Note: some of the syntax in the Guide is a bit out-dated, since yours truly has been slacking and haven't gotten everything updated yet 🙂 Verify with the reference docs to make sure the syntax is still valid)_

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:01pm UTC](https://discuss.elastic.co/t/boosting-a-field-yields-bizarre-results/46767/5 "2017-07-05T23:01:08Z")

</div>


