# Understanding the custom rescore plugin execution flow

**URL:** <https://discuss.elastic.co/t/understanding-the-custom-rescore-plugin-execution-flow/127776>\
**Category:** Elasticsearch\
**Created:** [April 12, 2018, 9:09am UTC](https://discuss.elastic.co/t/understanding-the-custom-rescore-plugin-execution-flow/127776 "2018-04-12T09:09:50Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![pere.urbon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pere.urbon/32/29099_2.png) [@pere.urbon](https://discuss.elastic.co/u/pere.urbon)\
**Post date:** [April 12, 2018, 9:09am UTC](https://discuss.elastic.co/t/understanding-the-custom-rescore-plugin-execution-flow/127776/1 "2018-04-12T09:09:50Z")

</div>

Hi,  
my name is Pere Urbon, I am currently building a plugin for elasticsearch to implement fair ranking/search strategies. My initial idea has been to build this as a recore plugin due to the fact that the original paper algorithm need all search results to be ready before the fair rescoring takes place.

I have created an initial version of the plugin, including few vanilla test using the great rest-api-spec test framework. In the test all works as expected, however when I run the plugin using kibana the rescorer does not get all the TopDocs search results at once, but in different waves. Does this makes sense? or I am doing something wrong?

I'm also wondering another question, taking into consideration that the algorithm needs all search results before doing the reordering, is it fair to implement it as a rescorer, or would be better as a custom query?

Cheers

-- Pere Urbon

---

<div class="post-metadata">

**Author:** ![dcausse](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dcausse/32/97582_2.png) [@dcausse](https://discuss.elastic.co/u/dcausse)\
**Post date:** [April 12, 2018, 1:04pm UTC](https://discuss.elastic.co/t/understanding-the-custom-rescore-plugin-execution-flow/127776/2 "2018-04-12T13:04:25Z")

</div>

Hi!

The rescore happens on each shard individually so you rerank the top-N of each shard. Let's say you have 2 shards you'll rescore 2\*top-N.  
If you can't adapt your algorithm to work this way and really want to work on the final top-N I'd suggest having a look at the solution implemented in [https://github.com/codelibs/elasticsearch-dynarank](https://github.com/codelibs/elasticsearch-dynarank) where the rescoring happens only once on the node that receives the request.

Note: While I don't know all the details, I'd suggest making your algorithm aware of this and potentially decrease its precision/exactness to be able to work on the top-N of each shards. Working on the final top-N sounds hard to me with challenging questions regarding performances and pagination.

---

<div class="post-metadata">

**Author:** ![pere.urbon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pere.urbon/32/29099_2.png) [@pere.urbon](https://discuss.elastic.co/u/pere.urbon)\
**Post date:** [April 14, 2018, 1:05pm UTC](https://discuss.elastic.co/t/understanding-the-custom-rescore-plugin-execution-flow/127776/3 "2018-04-14T13:05:03Z")

</div>

Thanks a lot David,  
stupid me, I should have thought that at the end the rescoring is like the query phase, happening at each shard. I find the approach interesting from your link, thanks a lot.

I might also work on adapting the method to work per shard, in terms of performance should be more efficient, however the work expects to be interesting 😸

Again, thank a lot,

-- Pere

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 12, 2018, 1:06pm UTC](https://discuss.elastic.co/t/understanding-the-custom-rescore-plugin-execution-flow/127776/4 "2018-05-12T13:06:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.