# Custom score query && pagination **URL:** \ **Category:** Elasticsearch\ **Created:** [November 18, 2013, 2:39pm UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450 "2013-11-18T14:39:25Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Yann\_Barraud](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yann_barraud/32/1108_2.png) [@Yann\_Barraud](https://discuss.elastic.co/u/Yann_Barraud)\ **Post date:** [November 18, 2013, 2:39pm UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/1 "2013-11-18T14:39:25Z")
Hi folks, I was wondering recently : how Elasticsearch perfoms on custom score queries & pagination ? Does it rescore the whole bunch each time ? Will it keep custom scores in cache ? Do I have to use a scroll id ? Thanks. Regards, Yann -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Nick\_Ethier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nick_ethier/32/1991_2.png) [@Nick\_Ethier](https://discuss.elastic.co/u/Nick_Ethier)\ **Post date:** [November 18, 2013, 5:53pm UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/2 "2013-11-18T17:53:42Z")
Hi Yann, Regardless of pagination or scrolling, the scores are always computed on the matching documents, keeping the scores in cache wouldn't be feasible and would require too much memory. On the shard level Elasticsearch will build a priority queue. For each matching document, Elasticsearch computes the score and updates the priority queue. When finished Elasticsearch takes the first "size" elements from the queue which are the hits. Hope this helps! Cheers. Nick On Mon, Nov 18, 2013 at 7:39 AM, Yann Barraud [yann.barraud@gmail.com](mailto:yann.barraud@gmail.com)wrote: > Hi folks, > > I was wondering recently : how Elasticsearch perfoms on custom score > queries & pagination ? Does it rescore the whole bunch each time ? Will it > keep custom scores in cache ? Do I have to use a scroll id ? > > Thanks. > > Regards, > Yann > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Yann\_Barraud](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yann_barraud/32/1108_2.png) [@Yann\_Barraud](https://discuss.elastic.co/u/Yann_Barraud)\ **Post date:** [November 19, 2013, 8:35am UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/3 "2013-11-19T08:35:06Z")
Hi nick, I does, for shure ! If I get things well, each page means a whole recalculation ? Cordialement, Yann Barraud 2013/11/18 Nick Ethier [ncethier@gmail.com](mailto:ncethier@gmail.com) > Hi Yann, > > Regardless of pagination or scrolling, the scores are always computed on > the matching documents, keeping the scores in cache wouldn't be feasible > and would require too much memory. > > On the shard level Elasticsearch will build a priority queue. For each > matching document, Elasticsearch computes the score and updates the > priority queue. When finished Elasticsearch takes the first "size" elements > from the queue which are the hits. > > Hope this helps! > > Cheers. > Nick > > On Mon, Nov 18, 2013 at 7:39 AM, Yann Barraud [yann.barraud@gmail.com](mailto:yann.barraud@gmail.com)wrote: > > > Hi folks, > > > > I was wondering recently : how Elasticsearch perfoms on custom score > > queries & pagination ? Does it rescore the whole bunch each time ? Will it > > keep custom scores in cache ? Do I have to use a scroll id ? > > > > Thanks. > > > > Regards, > > Yann > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/y0eFDXonNWQ/unsubscribe](https://groups.google.com/d/topic/elasticsearch/y0eFDXonNWQ/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [November 19, 2013, 9:58am UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/4 "2013-11-19T09:58:48Z")
Hi Yann, This is correct. The only way to not recompute everything between successive calls is to use the scan[1] search type (but it is limited: it doesn't allow for running facets or sorting for example). [1] > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. On Tue, Nov 19, 2013 at 9:35 AM, Yann Barraud [yann.barraud@gmail.com](mailto:yann.barraud@gmail.com)wrote: > Hi nick, > > I does, for shure ! > > If I get things well, each page means a whole recalculation ? > > Cordialement, > Yann Barraud > > 2013/11/18 Nick Ethier [ncethier@gmail.com](mailto:ncethier@gmail.com) > > > Hi Yann, > > > > Regardless of pagination or scrolling, the scores are always computed on > > the matching documents, keeping the scores in cache wouldn't be feasible > > and would require too much memory. > > > > On the shard level Elasticsearch will build a priority queue. For each > > matching document, Elasticsearch computes the score and updates the > > priority queue. When finished Elasticsearch takes the first "size" elements > > from the queue which are the hits. > > > > Hope this helps! > > > > Cheers. > > Nick > > > > On Mon, Nov 18, 2013 at 7:39 AM, Yann Barraud [yann.barraud@gmail.com](mailto:yann.barraud@gmail.com)wrote: > > > > > Hi folks, > > > > > > I was wondering recently : how Elasticsearch perfoms on custom score > > > queries & pagination ? Does it rescore the whole bunch each time ? Will it > > > keep custom scores in cache ? Do I have to use a scroll id ? > > > > > > Thanks. > > > > > > Regards, > > > Yann > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/y0eFDXonNWQ/unsubscribe](https://groups.google.com/d/topic/elasticsearch/y0eFDXonNWQ/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Yann\_Barraud](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yann_barraud/32/1108_2.png) [@Yann\_Barraud](https://discuss.elastic.co/u/Yann_Barraud)\ **Post date:** [November 19, 2013, 10:22am UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/5 "2013-11-19T10:22:06Z")
Thanks Adrien. The scroll search won't be better ? Le mardi 19 novembre 2013 10:58:48 UTC+1, Adrien Grand a écrit : > Hi Yann, > > This is correct. The only way to not recompute everything between > successive calls is to use the scan[1] search type (but it is limited: it > doesn't allow for running facets or sorting for example). > > [1] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-search-type.html#scan) > > On Tue, Nov 19, 2013 at 9:35 AM, Yann Barraud \<[yann.b...@gmail.com](mailto:yann.b...@gmail.com)\ > > > wrote: > > > Hi nick, > > > > I does, for shure ! > > > > If I get things well, each page means a whole recalculation ? > > > > Cordialement, > > Yann Barraud > > > > 2013/11/18 Nick Ethier \<[ncet...@gmail.com](mailto:ncet...@gmail.com) \\> > > > > > Hi Yann, > > > > > > Regardless of pagination or scrolling, the scores are always computed on > > > the matching documents, keeping the scores in cache wouldn't be feasible > > > and would require too much memory. > > > > > > On the shard level Elasticsearch will build a priority queue. For each > > > matching document, Elasticsearch computes the score and updates the > > > priority queue. When finished Elasticsearch takes the first "size" elements > > > from the queue which are the hits. > > > > > > Hope this helps! > > > > > > Cheers. > > > Nick > > > > > > On Mon, Nov 18, 2013 at 7:39 AM, Yann Barraud \<[yann.b...@gmail.com](mailto:yann.b...@gmail.com)\ > > > > > > > wrote: > > > > > > > Hi folks, > > > > > > > > I was wondering recently : how Elasticsearch perfoms on custom score > > > > queries & pagination ? Does it rescore the whole bunch each time ? Will it > > > > keep custom scores in cache ? Do I have to use a scroll id ? > > > > > > > > Thanks. > > > > > > > > Regards, > > > > Yann > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/y0eFDXonNWQ/unsubscribe](https://groups.google.com/d/topic/elasticsearch/y0eFDXonNWQ/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [November 19, 2013, 10:33am UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/6 "2013-11-19T10:33:24Z")
On Tue, Nov 19, 2013 at 11:22 AM, Yann Barraud [yann.barraud@gmail.com](mailto:yann.barraud@gmail.com)wrote: > Thanks Adrien. > > The scroll search won't be better ? The purpose of srolling is to execute queries that target different pages of the same query against the same point-in-time view of the index, so that results are not duplicated across pages and the number of total hits remains the same. This doesn't make things less costly. On the contrary, maintaining the contexts is expensive and we recommend to only use this feature when having consistent results across pages is a strong requirement. For example, this shouldn't be used on a high-traffic website to server user queries. -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Yann\_Barraud](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yann_barraud/32/1108_2.png) [@Yann\_Barraud](https://discuss.elastic.co/u/Yann_Barraud)\ **Post date:** [November 19, 2013, 1:16pm UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/7 "2013-11-19T13:16:51Z")
OK, thanks Adrien. Le mardi 19 novembre 2013 11:33:24 UTC+1, Adrien Grand a écrit : > On Tue, Nov 19, 2013 at 11:22 AM, Yann Barraud \<[yann.b...@gmail.com](mailto:yann.b...@gmail.com)\ > > > wrote: > > > Thanks Adrien. > > > > The scroll search won't be better ? > > The purpose of srolling is to execute queries that target different pages > of the same query against the same point-in-time view of the index, so that > results are not duplicated across pages and the number of total hits > remains the same. This doesn't make things less costly. On the contrary, > maintaining the contexts is expensive and we recommend to only use this > feature when having consistent results across pages is a strong > requirement. For example, this shouldn't be used on a high-traffic website > to server user queries. > > -- > Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:06am UTC](https://discuss.elastic.co/t/custom-score-query-pagination/14450/8 "2017-07-06T02:06:15Z")