# Items that should have equal score don't, could someone help me understand?

**URL:** <https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990>\
**Category:** Elasticsearch\
**Created:** [November 14, 2016, 1:57pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990 "2016-11-14T13:57:15Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 1:57pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/1 "2016-11-14T13:57:15Z")

</div>

Hi -- I have a query that searches for items in a certain subcategory that _should_ (not must) have been tagged with a certain tag. The items also have a `listed_at` field. If the query finds no items that have the desired tags, then it should just return the most recently listed items in the subcategory.  
Oh, and I'm using the `msearch` method to do a multi-search in the Ruby ElasticModel API.  
The query looks like this: [https://gist.github.com/biot023/11f12b82fb7df3dee816434f35b8021f](https://gist.github.com/biot023/11f12b82fb7df3dee816434f35b8021f)

I would expect that this would return the most recently listed items (according to the `listed_at` field), because I would expect all items to have the same score: they should all have the correct subcategory ID and they should none of them have the tag in the query ("snow leopard").  
However, the items seem to come back in a random order, each having been assigned a seemingly random score. An example of what actually gets returned is this: [https://gist.github.com/biot023/9a959631de8527ec9314e2f7ce34e2c7](https://gist.github.com/biot023/9a959631de8527ec9314e2f7ce34e2c7)

(Apologies for the massive code dump, I'm not sure what would be relevant or not.)  
As you can see, the scoring seems more-or-less random, which means that the `listed_at` field does not get to become the field sorted upon, as desired.

For completeness, the mappings of the items are as follows:

```
mappings do
  indexes( :title, type: "string", analyzer: "snowball" )
  indexes( :description, type: "string", analyzer: "snowball" )
  indexes( :tags, type: "string", index: "not_analyzed" )
  indexes( :section_id, type: "integer" )
  indexes( :category_id, type: "integer" )
  indexes( :subcategory_id, type: "integer" )
  indexes( :section, type: "string", index: "not_analyzed" )
  indexes( :category, type: "string", index: "not_analyzed" )
  indexes( :subcategory, type: "string", index: "not_analyzed" )
  indexes( :section_name, type: "string", analyzer: "snowball" )
  indexes( :category_name, type: "string", analyzer: "snowball" )
  indexes( :subcategory_name, type: "string", analyzer: "snowball" )
  indexes( :listed_at, type: "string", index: "not_analyzed" )
  indexes( :price, type: "float" )
  indexes( :price_range, type: "string", index: "not_analyzed" )
  indexes( :price_range_desc, type: "string", index: "not_analyzed" )
  indexes( :colour_ids, type: "integer" )
  indexes( :material_ids, type: "integer" )
  indexes( :created_at, type: "string", index: "not_analyzed" )
end

```

Could anyone explain how I'm ending up with a random scoring, and maybe explain how I can fix it so that the score stays consistent?  
Thankyou for any and all help offered,  
Doug.

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [November 14, 2016, 2:20pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/2 "2016-11-14T14:20:55Z")

</div>

Hi @DougFolksy,

one thing that jumps out is that you map `listed_at` as string. Your documents all have the same score (which is ok) but then Elasticsearch sorts _lexicographically_ by date (i.e. it treats the date as string) and I guess that's what you find so puzzling. So all you need to do is to map `listed_at` as date (and reindex the documents as the mapping does not apply to documents that already exist in the index).

Daniel

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:02pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/3 "2016-11-14T15:02:40Z")

</div>

Thanks for that @danielmitterdorfer  
Unfortunately, I get the same issue once I've changed the mapping to `date` and re-run my tests.  
For completeness, the mappings now look like this:

```
mappings do
  indexes( :title, type: "string", analyzer: "snowball" )
  indexes( :description, type: "string", analyzer: "snowball" )
  indexes( :tags, type: "string", index: "not_analyzed" )
  indexes( :section_id, type: "integer" )
  indexes( :category_id, type: "integer" )
  indexes( :subcategory_id, type: "integer" )
  indexes( :section, type: "string", index: "not_analyzed" )
  indexes( :category, type: "string", index: "not_analyzed" )
  indexes( :subcategory, type: "string", index: "not_analyzed" )
  indexes( :section_name, type: "string", analyzer: "snowball" )
  indexes( :category_name, type: "string", analyzer: "snowball" )
  indexes( :subcategory_name, type: "string", analyzer: "snowball" )
  indexes( :listed_at, type: "date", index: "not_analyzed" )
  indexes( :price, type: "float" )
  indexes( :price_range, type: "string", index: "not_analyzed" )
  indexes( :price_range_desc, type: "string", index: "not_analyzed" )
  indexes( :colour_ids, type: "integer" )
  indexes( :material_ids, type: "integer" )
  indexes( :created_at, type: "date", index: "not_analyzed" )
  indexes( :expires_at, type: "date", index: "not_analyzed" )
end

```

I tried it without the `index: "not_analyzed"` bit of the `date` mappings, too, but that didn't seem to make a difference.

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [November 14, 2016, 3:03pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/4 "2016-11-14T15:03:55Z")

</div>

Hi @DougFolksy,

did you recreate the index from scratch? Did you check the mapping and the documents directly in Elasticsearch?

Daniel

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:06pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/5 "2016-11-14T15:06:26Z")

</div>

Yes, @danielmitterdorfer, I deleted the index and re-imported.  
When I check my mappings, now, I see this for the `listed_at` field:

```
"listed_at" : {
  "type" : "date",
  "format" : "strict_date_optional_time||epoch_millis"
},
```

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:07pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/6 "2016-11-14T15:07:58Z")

</div>

Oh, and the results show items with `listed_at` dates like this:

```
"listed_at": "2016-11-13T23:07:03.000Z"

```

Which is different from before, so I'm guessing that that is correct.

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:12pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/7 "2016-11-14T15:12:01Z")

</div>

Ah!  
I think I might know what the cause is!  
The items that get lower scores than the others now seem to be ones with more tags.  
Could it be that they get a lower score because they have two tags that don't match the tag in the query? Whereas the other items that only have one tag that doesn't match get a higher score?  
In effect, the items are getting penalised for every tag they have which doesn't match the one in the query.  
Does that seem likely?

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:17pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/8 "2016-11-14T15:17:18Z")

</div>

Ah. No.  
That's not it.  
They're shifting around randomly, again.  
Sorry -- got a bit over-excited, there! ☹

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:29pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/9 "2016-11-14T15:29:50Z")

</div>

Just to confirm (sorry, I'm spamming aren't I?), the different items are getting different scores each time I run my tests -- here an example from two consecutive runs:

```
Item -- Score 1 -- Score 2
-----------------------------------
Item 01 -- 0.09848769 -- 0.04500804
Item 02 -- 0.28986934 -- 0.09848769
Item 03 -- 0.09848769 -- 0.09848769
Item 04 -- 0.28986934 -- 0.09848769
Item 05 -- 0.04500804 -- 0.04500804
Item 06 -- 0.04500804 -- 0.09848769

```

For each test run, the things that will differ (because its a different time, or they've been generated differently) will be:

- `listed_at` and other date fields ending in `_at`
- `_id` and `id`, as part of new test data being generated for the test
- `user_id`
- `shop_id`
- `section_id`
- `category_id`
- `subcategory_id`
- `primary_image` (to some random UUID)
- `section`, `category`, and `subcategory`, as these are fields composited from their IDs, slugs and names

But the only fields referenced in the query are `listed_at` and `tags` (an array of strings).  
So I can't help thinking that it must still be something to do with the `listed_at` field, as that's the only relevant thing that changes from test to test.

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [November 14, 2016, 3:32pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/10 "2016-11-14T15:32:36Z")

</div>

Hi @DougFolksy,

no worries. 🙂

You can use the [explain API](https://www.elastic.co/guide/en/elasticsearch/reference/master/search-explain.html) to find out how Elasticsearch calculated the score of a document.

Can you please also provide the search results as a gist? I'd like to see whether the results are now sorted by `_score` and then by date (but this time interpreted as date, not as string).

Regarding the random shuffling, can you please also provide the output of `GET /your_index_name_here/_stats`? (where you replace "your\_index\_name\_here" with the name of your index).

Daniel

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [November 14, 2016, 3:37pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/11 "2016-11-14T15:37:00Z")

</div>

Hi @DougFolksy,

you write:

> , the things that will differ (because its a different time, or they've been generated differently) will be:
> 
> - listed\_at and other date fields ending in \_at
> 
> [...]
> 
> the only fields referenced in the query are listed\_at [...]

Isn't it expected that you get different scores each time if you have different values for `listed_at` each time?

Daniel

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:38pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/12 "2016-11-14T15:38:35Z")

</div>

Sure thing, @danielmitterdorfer

The latest results are here: [https://gist.github.com/biot023/a10894b54c825d8cde4fc633082813b6](https://gist.github.com/biot023/a10894b54c825d8cde4fc633082813b6)

And the output from the stats query is here: [https://gist.github.com/biot023/3d3f4962afb4495de0180026d4126753](https://gist.github.com/biot023/3d3f4962afb4495de0180026d4126753)

I'm having a look at the explain API, now, and will report back -- thanks again, man!

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 3:41pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/13 "2016-11-14T15:41:16Z")

</div>

> Isn't it expected that you get different scores each time if you have different values for `listed_at` each time?

No, sorry, I should've explained -- the values change each time, but they're all consistently relative to each other.  
"Item 01" gets a `listed_at` time of 20 hours ago, "Item 02" gets one of 19 hours ago, and so on.

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [November 14, 2016, 3:46pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/14 "2016-11-14T15:46:01Z")

</div>

Hi @DougFolksy,

ok, I checked the results. You have a very small number of documents but multiple shards and this could cause (pseudo-)issues with scoring (see the section ["Relevance is broken" in the Definitive Guide](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/relevance-is-broken.html)). The gist of this: can you please create the index for your tests with only one primary shard, i.e.:

```auto
PUT /your_index_name
{ 
    "settings": { 
        "number_of_shards": 1 
    }
}

```

Daniel

---

<div class="post-metadata">

**Author:** ![DougFolksy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougfolksy/32/8658_2.png) [@DougFolksy](https://discuss.elastic.co/u/DougFolksy)\
**Post date:** [November 14, 2016, 4:04pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/15 "2016-11-14T16:04:13Z")

</div>

When I get around to writing the epic opera based upon this thread, your character's arias are going to be **so** heroic!  
It took me a while how to figure out how to do this with our test setup, but once figured, I've not had an inconsistent result, yet!  
Thankyou so much, @danielmitterdorfer!

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [November 14, 2016, 4:05pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/16 "2016-11-14T16:05:21Z")

</div>

Haha, glad I could help you @DougFolksy and I'm looking forward to your opera. In the meantime, enjoy your now consistent search results. 🙂

Daniel

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 12, 2016, 4:05pm UTC](https://discuss.elastic.co/t/items-that-should-have-equal-score-dont-could-someone-help-me-understand/65990/17 "2016-12-12T16:05:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.