# Inconsistent paging **URL:** \ **Category:** Elasticsearch\ **Created:** [August 18, 2014, 6:02am UTC](https://discuss.elastic.co/t/inconsistent-paging/19315 "2014-08-18T06:02:08Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Ron\_Sher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ron_sher/32/4472_2.png) [@Ron\_Sher](https://discuss.elastic.co/u/Ron_Sher)\ **Post date:** [August 18, 2014, 6:02am UTC](https://discuss.elastic.co/t/inconsistent-paging/19315/1 "2014-08-18T06:02:08Z")
Hi, We've noticed a strange behavior in elasticsearch during paging. In one case we use a paging size of 60 and we have 63 documents. So the first page is using size 60 and offset 0. The second page is using size 60 and offset 60. What we see is that the result is inconsistent. Meaning, on the 2nd page, we sometimes get results that were before in the 1st page. The query we use has an order by some numberic field that has many documents with the same value (0). It looks like the ordering between documents according to the same value, which is 0, isn't consistent. Did anyone encounter such behavior? Any suggestions on resolving this? We're using version 1.3.1. Thanks, Ron -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [August 18, 2014, 7:44am UTC](https://discuss.elastic.co/t/inconsistent-paging/19315/2 "2014-08-18T07:44:54Z")
You need to use scroll if you have that requirement. See: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html#search-request-scroll) -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 18 août 2014 à 08:02, Ron Sher [ron.sher@gmail.com](mailto:ron.sher@gmail.com) a écrit : > > Hi, > > We've noticed a strange behavior in elasticsearch during paging. > > In one case we use a paging size of 60 and we have 63 documents. So the first page is using size 60 and offset 0. The second page is using size 60 and offset 60. What we see is that the result is inconsistent. Meaning, on the 2nd page, we sometimes get results that were before in the 1st page. > > The query we use has an order by some numberic field that has many documents with the same value (0). > It looks like the ordering between documents according to the same value, which is 0, isn't consistent. > > Did anyone encounter such behavior? Any suggestions on resolving this? > > We're using version 1.3.1. > > ## Thanks, Ron > > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8DAEA97B-687A-44A6-B638-189A49D6310E%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/8DAEA97B-687A-44A6-B638-189A49D6310E%40pilato.fr). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [August 18, 2014, 8:02am UTC](https://discuss.elastic.co/t/inconsistent-paging/19315/3 "2014-08-18T08:02:29Z")
Hi Ron, The cause of this issue is that Elasticsearch uses Lucene's internal doc IDs as tie-breakers. Internal doc IDs might be completely different across replicas of the same data, so this explains why documents that have the same sort values are not consistently ordered. There are 2 potential ways to fix that problem: 1. Use scroll as David mentionned. It will create a context around your request and will make sure that the same shards will be used for all pages. However, it also gives another warranty, which is that the same point-in-time view on the index will be used for each page, and this is expensive to maintain. 2. Use a custom string value as a preference in order to always hit the same shards for a given session[1]. This will help with always hitting the same shards likely to 1. but without adding the additional cost of a scroll. [1] > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. On Mon, Aug 18, 2014 at 8:02 AM, Ron Sher [ron.sher@gmail.com](mailto:ron.sher@gmail.com) wrote: > Hi, > > We've noticed a strange behavior in elasticsearch during paging. > > In one case we use a paging size of 60 and we have 63 documents. So the > first page is using size 60 and offset 0. The second page is using size 60 > and offset 60. What we see is that the result is inconsistent. Meaning, > on the 2nd page, we sometimes get results that were before in the 1st page. > > The query we use has an order by some numberic field that has many > documents with the same value (0). > It looks like the ordering between documents according to the same value, > which is 0, isn't consistent. > > Did anyone encounter such behavior? Any suggestions on resolving this? > > We're using version 1.3.1. > > Thanks, > Ron > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7FJofXSpDjHnpMVs1poHFREbrQ9DPnPX4YnjFjUKg\_ng%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7FJofXSpDjHnpMVs1poHFREbrQ9DPnPX4YnjFjUKg_ng%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ron\_Sher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ron_sher/32/4472_2.png) [@Ron\_Sher](https://discuss.elastic.co/u/Ron_Sher)\ **Post date:** [August 25, 2014, 4:10pm UTC](https://discuss.elastic.co/t/inconsistent-paging/19315/4 "2014-08-25T16:10:43Z")
Thanks for the answer and sorry for the duplicate (posted from a different source by mistake) On Monday, August 18, 2014 11:02:47 AM UTC+3, Adrien Grand wrote: > Hi Ron, > > The cause of this issue is that Elasticsearch uses Lucene's internal doc > IDs as tie-breakers. Internal doc IDs might be completely different > across replicas of the same data, so this explains why documents that have > the same sort values are not consistently ordered. > > There are 2 potential ways to fix that problem: > > 1. Use scroll as David mentionned. It will create a context around your > request and will make sure that the same shards will be used for all pages. > However, it also gives another warranty, which is that the same > point-in-time view on the index will be used for each page, and this is > expensive to maintain. > 2. Use a custom string value as a preference in order to always hit the > same shards for a given session[1]. This will help with always hitting > the same shards likely to 1. but without adding the additional cost of a > scroll. > > [1] > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-preference.html) > > On Mon, Aug 18, 2014 at 8:02 AM, Ron Sher \<[ron....@gmail.com](mailto:ron....@gmail.com) \ > > > wrote: > > > Hi, > > > > We've noticed a strange behavior in elasticsearch during paging. > > > > In one case we use a paging size of 60 and we have 63 documents. So the > > first page is using size 60 and offset 0. The second page is using size 60 > > and offset 60. What we see is that the result is inconsistent. Meaning, > > on the 2nd page, we sometimes get results that were before in the 1st page. > > > > The query we use has an order by some numberic field that has many > > documents with the same value (0). > > It looks like the ordering between documents according to the same value, > > which is 0, isn't consistent. > > > > Did anyone encounter such behavior? Any suggestions on resolving this? > > > > We're using version 1.3.1. > > > > Thanks, > > Ron > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAKHuyJpcYKepYzh%2BBU2MSD2RQ19zjHYiXgf3anWBL9esq9fkGQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > Adrien Grand -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/277ec3ee-f7bf-4862-a816-efe2937a9609%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/277ec3ee-f7bf-4862-a816-efe2937a9609%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:06am UTC](https://discuss.elastic.co/t/inconsistent-paging/19315/5 "2017-07-06T01:06:39Z")