# Do unique/reusable \_scroll\_ids exist?
**URL:** https://discuss.elastic.co/t/do-unique-reusable--scroll-ids-exist/12280
**Category:** Elasticsearch
**Created:** [June 5, 2013, 3:48pm UTC](https://discuss.elastic.co/t/do-unique-reusable--scroll-ids-exist/12280 "2013-06-05T15:48:43Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Oli\_McCormack](https://discuss.elastic.co/u/Oli_McCormack)
#### Post date: [June 5, 2013, 3:48pm UTC](https://discuss.elastic.co/t/do-unique-reusable--scroll-ids-exist/12280/1 "2013-06-05T15:48:43Z")
Hi guys,
I'm attempting to implement pagination for our application. The catch is
that our documents require a little post-query filtering, so sometimes if a
user requests 500 documents, we scroll, get 500 from ES, filter and end up
with a lower number. In this case, we perform the next scroll, get a number
of results and build until we have 500 valid docs.
I had some related questions about scrolling / the scroll id returned by
search scroll requests.
_Question1_: Is it possible to use the same scroll id multiple times to get
the same set of results in the over-all result set?
_Question2_: (related to Question1) I'm confused by the scroll\_id returned
whilst doing a scan search then scrolling. What I see is that that when I
start scrolling, for a period of time I get the same \_scroll\_id back. After
some number of requests it changes. I would have expected to either (1) get
the same \_scroll\_id over and over or (2) get a different \_scroll\_id each
time. Are either of these correct? At the bottom of this mail I've given a
short example set of req/resp.
Any pointers on this appreciated. I'd also be interested in hearing from
anyone who has successfully implemented pagination and the approach you
took.
Cheers,
oli
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Oli\_McCormack](https://discuss.elastic.co/u/Oli_McCormack)
#### Post date: [June 5, 2013, 3:50pm UTC](https://discuss.elastic.co/t/do-unique-reusable--scroll-ids-exist/12280/2 "2013-06-05T15:50:34Z")
Ah, I never gave my example. In case it's of use:
_Request 1_
> > curl -XPOST
> > 'localhost:9200/foo/bar/\_search?search\_type=scan&scroll=10m&size=10'
> > -d
> > '{"query":{"constant\_score":{"boost":1,"filter":{"term":{"x":false}}}}}'
{"\_scroll\_id":"_abc_
","took":1,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":108,"max\_score":0.0,"hits":}}
_Request 2_
> > curl -XPOST 'localhost:9200/\_search/scroll?scroll=10m'
> > -d '_abc_'
{"\_scroll\_id":"_abc_","took":1,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":108,"max\_score":0.0,"hits":[{..},
{..}]}}
_.. after some number of requests_
{"\_scroll\_id":"_def_","took":1,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":108,"max\_score":0.0,"hits":[{..},
{..}]}}
On Wednesday, June 5, 2013 8:48:43 AM UTC-7, Oli wrote:
> Hi guys,
>
> I'm attempting to implement pagination for our application. The catch is
> that our documents require a little post-query filtering, so sometimes if a
> user requests 500 documents, we scroll, get 500 from ES, filter and end up
> with a lower number. In this case, we perform the next scroll, get a number
> of results and build until we have 500 valid docs.
>
> I had some related questions about scrolling / the scroll id returned by
> search scroll requests.
>
> _Question1_: Is it possible to use the same scroll id multiple times to
> get the same set of results in the over-all result set?
>
> _Question2_: (related to Question1) I'm confused by the scroll\_id
> returned whilst doing a scan search then scrolling. What I see is that
> that when I start scrolling, for a period of time I get the same \_scroll\_id
> back. After some number of requests it changes. I would have expected to
> either (1) get the same \_scroll\_id over and over or (2) get a different
> \_scroll\_id each time. Are either of these correct? At the bottom of this
> mail I've given a short example set of req/resp.
>
> Any pointers on this appreciated. I'd also be interested in hearing from
> anyone who has successfully implemented pagination and the approach you
> took.
>
> Cheers,
> oli
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@sujoysett](https://discuss.elastic.co/u/sujoysett)
#### Post date: [June 10, 2013, 9:27am UTC](https://discuss.elastic.co/t/do-unique-reusable--scroll-ids-exist/12280/3 "2013-06-10T09:27:31Z")
Hi,
I had faced this issue quite long ago while implementing custom client side
code for ES data backup and re-indexing purpose.
Yes, the scroll ID remains same for few hits, after which it changes.
The solution is to use the scroll ID returned with every hit response in
the subsequent request, i.e. following an ID chaining mechanism will work.
Using the first scroll ID repeatedly fetches only a few results, not all.
I guess the scroll ID gets renewed after the timestamp expires (calculated
from the point of first hit). But this statement is based on random
observation, I am not sure of this, ES experts can elaborate the underlying
cause better. I would be glad to know the actual cause too.
- Sujoy.
On Wednesday, June 5, 2013 9:20:34 PM UTC+5:30, Oli wrote:
> Ah, I never gave my example. In case it's of use:
>
> _Request 1_
>
> > > curl -XPOST
> > > 'localhost:9200/foo/bar/\_search?search\_type=scan&scroll=10m&size=10'
> > > -d
> > > '{"query":{"constant\_score":{"boost":1,"filter":{"term":{"x":false}}}}}'
>
> {"\_scroll\_id":"_abc_
> ","took":1,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":108,"max\_score":0.0,"hits":}}
>
> _Request 2_
>
> > > curl -XPOST 'localhost:9200/\_search/scroll?scroll=10m'
> > > -d '_abc_'
>
> {"\_scroll\_id":"_abc_","took":1,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":108,"max\_score":0.0,"hits":[{..},
> {..}]}}
>
> _.. after some number of requests_
>
> {"\_scroll\_id":"_def_","took":1,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":108,"max\_score":0.0,"hits":[{..},
> {..}]}}
>
> On Wednesday, June 5, 2013 8:48:43 AM UTC-7, Oli wrote:
>
> > Hi guys,
> >
> > I'm attempting to implement pagination for our application. The catch is
> > that our documents require a little post-query filtering, so sometimes if a
> > user requests 500 documents, we scroll, get 500 from ES, filter and end up
> > with a lower number. In this case, we perform the next scroll, get a number
> > of results and build until we have 500 valid docs.
> >
> > I had some related questions about scrolling / the scroll id returned by
> > search scroll requests.
> >
> > _Question1_: Is it possible to use the same scroll id multiple times to
> > get the same set of results in the over-all result set?
> >
> > _Question2_: (related to Question1) I'm confused by the scroll\_id
> > returned whilst doing a scan search then scrolling. What I see is that
> > that when I start scrolling, for a period of time I get the same \_scroll\_id
> > back. After some number of requests it changes. I would have expected to
> > either (1) get the same \_scroll\_id over and over or (2) get a different
> > \_scroll\_id each time. Are either of these correct? At the bottom of this
> > mail I've given a short example set of req/resp.
> >
> > Any pointers on this appreciated. I'd also be interested in hearing from
> > anyone who has successfully implemented pagination and the approach you
> > took.
> >
> > Cheers,
> > oli
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Oli\_McCormack](https://discuss.elastic.co/u/Oli_McCormack)
#### Post date: [June 10, 2013, 6:12pm UTC](https://discuss.elastic.co/t/do-unique-reusable--scroll-ids-exist/12280/4 "2013-06-10T18:12:27Z")
Thanks Sujoy, appreciate you getting back to this.
I've found that to be the solution. The unfortunate thing for me is that
I'd like to able to re-fetch the results for a specific position in a
scroll.
Given that a token appears to always yield the \*next \*set of results (at
least for all of the results that token represents) it seems like I can't
ever re-fetch something I've already obtained once. Any information to the
contrary of that would be great to hear!
- oli
On Mon, Jun 10, 2013 at 2:27 AM, Sujoy Sett [sujoysett@gmail.com](mailto:sujoysett@gmail.com) wrote:
> Hi,
>
> I had faced this issue quite long ago while implementing custom client
> side code for ES data backup and re-indexing purpose.
> Yes, the scroll ID remains same for few hits, after which it changes.
>
> The solution is to use the scroll ID returned with every hit response in
> the subsequent request, i.e. following an ID chaining mechanism will work.
>
> Using the first scroll ID repeatedly fetches only a few results, not all.
> I guess the scroll ID gets renewed after the timestamp expires (calculated
> from the point of first hit). But this statement is based on random
> observation, I am not sure of this, ES experts can elaborate the underlying
> cause better. I would be glad to know the actual cause too.
>
> - Sujoy.
>
> On Wednesday, June 5, 2013 9:20:34 PM UTC+5:30, Oli wrote:
>
> > Ah, I never gave my example. In case it's of use:
> >
> > _Request 1_
> >
> > > > curl -XPOST 'localhost:9200/foo/bar/\_\*\*search?search\_type=scan&\*\*scroll=10m&size=10'
> > > >
> > > > -d '{"query":{"constant\_score":{"\*\*boost":1,"filter":{"term":{"x"
> > > > \*\*:false}}}}}'
> >
> > {"\_scroll\_id":"_abc_","took":1," **timed\_out":false,"\_shards":{"**
> > total":5,"successful":5," **failed":0},"hits":{"total":**
> > 108,"max\_score":0.0,"hits":}\*\*}
> >
> > _Request 2_
> >
> > > > curl -XPOST 'localhost:9200/\_search/\*\*scroll?scroll=10m'
> > > > -d '_abc_'
> >
> > {"\_scroll\_id":"_abc_","took":1," **timed\_out":false,"\_shards":{"**
> > total":5,"successful":5," **failed":0},"hits":{"total":**
> > 108,"max\_score":0.0,"hits":[{.\*\*.}, {..}]}}
> >
> > _.. after some number of requests_
> >
> > {"\_scroll\_id":"_def_","took":1," **timed\_out":false,"\_shards":{"**
> > total":5,"successful":5," **failed":0},"hits":{"total":**
> > 108,"max\_score":0.0,"hits":[{.\*\*.}, {..}]}}
> >
> > On Wednesday, June 5, 2013 8:48:43 AM UTC-7, Oli wrote:
> >
> > > Hi guys,
> > >
> > > I'm attempting to implement pagination for our application. The catch
> > > is that our documents require a little post-query filtering, so sometimes
> > > if a user requests 500 documents, we scroll, get 500 from ES, filter and
> > > end up with a lower number. In this case, we perform the next scroll, get a
> > > number of results and build until we have 500 valid docs.
> > >
> > > I had some related questions about scrolling / the scroll id returned by
> > > search scroll requests.
> > >
> > > _Question1_: Is it possible to use the same scroll id multiple times to
> > > get the same set of results in the over-all result set?
> > >
> > > _Question2_: (related to Question1) I'm confused by the scroll\_id
> > > returned whilst doing a scan search then scrolling. What I see is that
> > > that when I start scrolling, for a period of time I get the same \_scroll\_id
> > > back. After some number of requests it changes. I would have expected to
> > > either (1) get the same \_scroll\_id over and over or (2) get a different
> > > \_scroll\_id each time. Are either of these correct? At the bottom of this
> > > mail I've given a short example set of req/resp.
> > >
> > > Any pointers on this appreciated. I'd also be interested in hearing from
> > > anyone who has successfully implemented pagination and the approach you
> > > took.
> > >
> > > Cheers,
> > > oli
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:31am UTC](https://discuss.elastic.co/t/do-unique-reusable--scroll-ids-exist/12280/5 "2017-07-06T02:31:58Z")