# Pagination with range queries giving duplicate results
**URL:** https://discuss.elastic.co/t/pagination-with-range-queries-giving-duplicate-results/21893
**Category:** Elasticsearch
**Created:** [January 29, 2015, 7:06am UTC](https://discuss.elastic.co/t/pagination-with-range-queries-giving-duplicate-results/21893 "2015-01-29T07:06:24Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@asthanaamish](https://discuss.elastic.co/u/asthanaamish)
#### Post date: [January 29, 2015, 7:06am UTC](https://discuss.elastic.co/t/pagination-with-range-queries-giving-duplicate-results/21893/1 "2015-01-29T07:06:24Z")
Hi Folks
We are facing an issue intermittently of range queries with pagination
missing some records or giving duplicate ids.
Let me describe our system.
Lets say we have certain number of records and during which queries are
being made in ES, we can assume that no record is getting created/updated
or deleted.
The query we have is something like :
{"query":{"bool":{"must":[{"range":{"lastname":{"from":"Doe","to":null,"include\_lower":false,"include\_upper":true}}},{"range":{"firstname":{"from":"joe","to":null,"include\_lower":false,"include\_upper":true}}}]}}}
Now we have also have from":X,"size":Y, and we will issue multiple queries
with from increasing every time as X = X + Y.
The idea is that every time we will get unique records.
Unfortunately from time to time it does not happen. ( As I said its a
closed system, so lets assume nobody is updating/deleting/creating data.)
We see some records which are not there and some records which are
duplicate.
Anybody has seen similar issue, or can shed some light as to how we should
debug this?
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c5bb300a-ee7c-48d5-a408-f9a1a1b267b6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c5bb300a-ee7c-48d5-a408-f9a1a1b267b6%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [January 29, 2015, 7:21am UTC](https://discuss.elastic.co/t/pagination-with-range-queries-giving-duplicate-results/21893/2 "2015-01-29T07:21:10Z")
Do you get duplicates within the same page or only in another page?
To ensure consistent pagination, I would use scroll API. Could you try with it?
--
David 😉
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
> Le 29 janv. 2015 à 08:06, Amish Asthana [asthanaamish@gmail.com](mailto:asthanaamish@gmail.com) a écrit :
>
> Hi Folks
> We are facing an issue intermittently of range queries with pagination missing some records or giving duplicate ids.
>
> Let me describe our system.
> Lets say we have certain number of records and during which queries are being made in ES, we can assume that no record is getting created/updated or deleted.
>
> The query we have is something like :
> {"query":{"bool":{"must":[{"range":{"lastname":{"from":"Doe","to":null,"include\_lower":false,"include\_upper":true}}},{"range":{"firstname":{"from":"joe","to":null,"include\_lower":false,"include\_upper":true}}}]}}}
>
> Now we have also have from":X,"size":Y, and we will issue multiple queries with from increasing every time as X = X + Y.
>
> The idea is that every time we will get unique records.
> Unfortunately from time to time it does not happen. ( As I said its a closed system, so lets assume nobody is updating/deleting/creating data.)
> We see some records which are not there and some records which are duplicate.
>
> Anybody has seen similar issue, or can shed some light as to how we should debug this?
>
> --
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/c5bb300a-ee7c-48d5-a408-f9a1a1b267b6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/c5bb300a-ee7c-48d5-a408-f9a1a1b267b6%40googlegroups.com).
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7F981207-6D46-4973-87BD-B5C2D5E4702A%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/7F981207-6D46-4973-87BD-B5C2D5E4702A%40pilato.fr).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@asthanaamish](https://discuss.elastic.co/u/asthanaamish)
#### Post date: [January 29, 2015, 2:50pm UTC](https://discuss.elastic.co/t/pagination-with-range-queries-giving-duplicate-results/21893/3 "2015-01-29T14:50:21Z")
Hi David
We are aware of scroll API, and are not using it as it will not scale.
That is the very reason I was stressing the fact that there is no
update/delete/create; as with multiple queries all bets are off if any of
this thing happen.
However with steady state)no change in data) I would expect them to work.
To answer your question : It happens in different pagination group.
regards and thanks
amish
On Wednesday, January 28, 2015 at 11:06:24 PM UTC-8, Amish Asthana wrote:
> Hi Folks
> We are facing an issue intermittently of range queries with pagination
> missing some records or giving duplicate ids.
>
> Let me describe our system.
> Lets say we have certain number of records and during which queries are
> being made in ES, we can assume that no record is getting created/updated
> or deleted.
>
> The query we have is something like :
>
> {"query":{"bool":{"must":[{"range":{"lastname":{"from":"Doe","to":null,"include\_lower":false,"include\_upper":true}}},{"range":{"firstname":{"from":"joe","to":null,"include\_lower":false,"include\_upper":true}}}]}}}
>
> Now we have also have from":X,"size":Y, and we will issue multiple queries
> with from increasing every time as X = X + Y.
>
> The idea is that every time we will get unique records.
> Unfortunately from time to time it does not happen. ( As I said its a
> closed system, so lets assume nobody is updating/deleting/creating data.)
> We see some records which are not there and some records which are
> duplicate.
>
> Anybody has seen similar issue, or can shed some light as to how we should
> debug this?
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e005e536-5406-4321-b736-93b200765393%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e005e536-5406-4321-b736-93b200765393%40googlegroups.com).
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:35am UTC](https://discuss.elastic.co/t/pagination-with-range-queries-giving-duplicate-results/21893/4 "2017-07-06T00:35:55Z")