Hey there!
I have an interesting problem that I do have a hunch on but lack a full explanation. I am querying an index with a solid double digit million number of documents.
One of my queries (nested, function score, scripts, regular filters, quite a bit of things) runs against 17k documents and takes between 16-25ms on average, which is awesome.
How I run another query, very similar to this one, with an additional filter, that reduces the results down to about 3k (sometimes even less). So my natural expectation was, that it would be at the same speed or even faster. Turns out it's slower by a factor of 2 on average with much slower outliers like 80ms. The difference is sorting. One query sorts by the score, the slower one by score and an additional criteria (that is a unique ID in our dataset to ensure that same scores produce consistent order).
"sort": [
{
"_score": "desc"
},
{
"my_field": { "order": "desc"}
}
]
I am aware that this needs to be loaded into field data, but the field is already loaded eagerly and also using _doc or _seq_no as field does not speed it up.
What am I missing to understand why is this so inefficient, if it is just supposed to be a tie breaker? This query returns the top 50 results and the tie breaker is actually not needed in this query, as all scores are different.
This is on Elasticsearch 8.14.
Thanks for any pointers!
--Alex