# Distributing query results (using scrolling?)
**URL:** https://discuss.elastic.co/t/distributing-query-results-using-scrolling/12448
**Category:** Elasticsearch
**Created:** [June 17, 2013, 4:28pm UTC](https://discuss.elastic.co/t/distributing-query-results-using-scrolling/12448 "2013-06-17T16:28:18Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Alex\_At\_Ikanow](https://discuss.elastic.co/u/Alex_At_Ikanow)
#### Post date: [June 17, 2013, 4:28pm UTC](https://discuss.elastic.co/t/distributing-query-results-using-scrolling/12448/1 "2013-06-17T16:28:18Z")
I've been thinking about ways of running Hadoop jobs on the results of
elasticsearch queries.
I was hoping that I could just distribute a scrollid to the different
mappers and then have them read and process batches of data served as
needed from their closest es node (data locality isn't really an issue
here, the hadoop and es nodes will normally not be co-located)
The documentation is pretty clear that this shouldn't work (
[http://www.elasticsearch.org/guide/reference/api/search/scroll/](http://www.elasticsearch.org/guide/reference/api/search/scroll/)) "_Note_:
the scroll\_id changes for each scroll request and only the most recent one
should be used" ... but running on 0.19, the scroll id did appear to remain
the same .... does anybody know which of the following is true:
- On all versions the scrollid can change, though sometimes won't
- It didn't change up until version XXX, but now does
- It doesn't change, but we put that in because we might need to make it
change in the future?
(It would also require the get-more-data operation to be atomic across
multiple nodes, which not be the case)
Any pointers to good spots in the codebase to look at the inner workings of
scrolls? (I always get a bit lost in the middle of the es code!)
Thanks for any insight/pointers
Alex
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Oli\_McCormack](https://discuss.elastic.co/u/Oli_McCormack)
#### Post date: [June 17, 2013, 4:50pm UTC](https://discuss.elastic.co/t/distributing-query-results-using-scrolling/12448/2 "2013-06-17T16:50:23Z")
Hi Alex,
I've recently been evaluating a very similar use case, you might be
interested in these threads (unique scroll
ids[https://groups.google.com/forum/#!topic/elasticsearch/tqmwhuDADFw](https://groups.google.com/forum/#!topic/elasticsearch/tqmwhuDADFw)and
characteristics
of scrolling[https://groups.google.com/forum/#!topic/elasticsearch/55ByVEr93bo](https://groups.google.com/forum/#!topic/elasticsearch/55ByVEr93bo)
).
I can't speak to different versions of ES or codebase specifics, but my
guess is that what you're seeing is similar to what I see in 0.20.4:
- A \_scroll\_id will remain the same for some number of requests issued
against it, and at some point will change.
- Issuing multiple requests against the same \_scroll\_id will yield
subsequent results (at least for the overall resultset that that \_scroll\_id
represents before it changes), not the same results.
- You must use the most recent \_scroll\_id (as you mentioned) to be sure
to accurately get the full set of results for the query.
If you're not seeing the scroll id change in 0.19. I would guess that your
query is small enough that it doesn't end up giving you a new id, but
that's just a hunch.
To conclude, I don't \*think \*you'll be able to achieve what you want using
scrollers (at least that's the conclusion I've come to - I'd love to hear
otherwise). An alternative approach is using from/to attributes of a normal
query and have each mapper operate on a certain range of a query.
- oli
On Mon, Jun 17, 2013 at 9:28 AM, Alex at Ikanow [apiggott@ikanow.com](mailto:apiggott@ikanow.com) wrote:
> I've been thinking about ways of running Hadoop jobs on the results of
> elasticsearch queries.
>
> I was hoping that I could just distribute a scrollid to the different
> mappers and then have them read and process batches of data served as
> needed from their closest es node (data locality isn't really an issue
> here, the hadoop and es nodes will normally not be co-located)
>
> The documentation is pretty clear that this shouldn't work (
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/scroll/)) "_Note_:
> the scroll\_id changes for each scroll request and only the most recent
> one should be used" ... but running on 0.19, the scroll id did appear to
> remain the same .... does anybody know which of the following is true:
>
> - On all versions the scrollid can change, though sometimes won't
> - It didn't change up until version XXX, but now does
> - It doesn't change, but we put that in because we might need to make it
> change in the future?
>
> (It would also require the get-more-data operation to be atomic across
> multiple nodes, which not be the case)
>
> Any pointers to good spots in the codebase to look at the inner workings
> of scrolls? (I always get a bit lost in the middle of the es code!)
>
> Thanks for any insight/pointers
>
> Alex
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Alex\_At\_Ikanow](https://discuss.elastic.co/u/Alex_At_Ikanow)
#### Post date: [June 17, 2013, 6:24pm UTC](https://discuss.elastic.co/t/distributing-query-results-using-scrolling/12448/3 "2013-06-17T18:24:13Z")
Hey thanks for sharing your experiences and those links
If there's 2 of us then maybe that's enough for a feature request 🙂
Hmm looking at the code, it seems like all the scroll function does is save
and apply the from/to information anyway (
[https://github.com/elasticsearch/elasticsearch/blob/f09ad507a44367ed6ca29c6f3dae7659a2da1994/src/main/java/org/elasticsearch/search/SearchService.java](https://github.com/elasticsearch/elasticsearch/blob/f09ad507a44367ed6ca29c6f3dae7659a2da1994/src/main/java/org/elasticsearch/search/SearchService.java)),
so no reason not to do as you suggest!
(The existing elasticsearch-hadoop connector incidentally generates 1
mapper per shard and scrolls through that, which might be as well as you
can do anyway)
On Monday, June 17, 2013 12:50:23 PM UTC-4, Oli wrote:
> Hi Alex,
>
> I've recently been evaluating a very similar use case, you might be
> interested in these threads (unique scroll ids[https://groups.google.com/forum/#!topic/elasticsearch/tqmwhuDADFw](https://groups.google.com/forum/#!topic/elasticsearch/tqmwhuDADFw)and characteristics
> of scrolling[https://groups.google.com/forum/#!topic/elasticsearch/55ByVEr93bo](https://groups.google.com/forum/#!topic/elasticsearch/55ByVEr93bo)
> ).
>
> I can't speak to different versions of ES or codebase specifics, but my
> guess is that what you're seeing is similar to what I see in 0.20.4:
>
> - A \_scroll\_id will remain the same for some number of requests issued
> against it, and at some point will change.
> - Issuing multiple requests against the same \_scroll\_id will yield
> subsequent results (at least for the overall resultset that that \_scroll\_id
> represents before it changes), not the same results.
> - You must use the most recent \_scroll\_id (as you mentioned) to be
> sure to accurately get the full set of results for the query.
>
> If you're not seeing the scroll id change in 0.19. I would guess that your
> query is small enough that it doesn't end up giving you a new id, but
> that's just a hunch.
>
> To conclude, I don't \*think \*you'll be able to achieve what you want
> using scrollers (at least that's the conclusion I've come to - I'd love to
> hear otherwise). An alternative approach is using from/to attributes of a
> normal query and have each mapper operate on a certain range of a query.
>
> - oli
>
> On Mon, Jun 17, 2013 at 9:28 AM, Alex at Ikanow \<[apig...@ikanow.com](mailto:apig...@ikanow.com)\
>
> > wrote:
>
> > I've been thinking about ways of running Hadoop jobs on the results of
> > elasticsearch queries.
> >
> > I was hoping that I could just distribute a scrollid to the different
> > mappers and then have them read and process batches of data served as
> > needed from their closest es node (data locality isn't really an issue
> > here, the hadoop and es nodes will normally not be co-located)
> >
> > The documentation is pretty clear that this shouldn't work (
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/scroll/)) "_Note_:
> > the scroll\_id changes for each scroll request and only the most recent
> > one should be used" ... but running on 0.19, the scroll id did appear to
> > remain the same .... does anybody know which of the following is true:
> >
> > - On all versions the scrollid can change, though sometimes won't
> > - It didn't change up until version XXX, but now does
> > - It doesn't change, but we put that in because we might need to make it
> > change in the future?
> >
> > (It would also require the get-more-data operation to be atomic across
> > multiple nodes, which not be the case)
> >
> > Any pointers to good spots in the codebase to look at the inner workings
> > of scrolls? (I always get a bit lost in the middle of the es code!)
> >
> > Thanks for any insight/pointers
> >
> > Alex
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send an
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \.
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Alex\_At\_Ikanow](https://discuss.elastic.co/u/Alex_At_Ikanow)
#### Post date: [June 17, 2013, 6:31pm UTC](https://discuss.elastic.co/t/distributing-query-results-using-scrolling/12448/4 "2013-06-17T18:31:57Z")
(Clarification: I'm guessing it caches a block of matching \_ids then
scrolls through that using the from/to construct until you reach the end
and then it does another query, which is when the scrollid changes, or
something like that. So the problem with multiple to/from queries is that
you'll redo the same query over and over, although some subset of it will
be cached so will run more quickly subsequent times)
On Monday, June 17, 2013 2:24:13 PM UTC-4, Alex at Ikanow wrote:
> Hey thanks for sharing your experiences and those links
>
> If there's 2 of us then maybe that's enough for a feature request 🙂
>
> Hmm looking at the code, it seems like all the scroll function does is
> save and apply the from/to information anyway (
> [https://github.com/elasticsearch/elasticsearch/blob/f09ad507a44367ed6ca29c6f3dae7659a2da1994/src/main/java/org/elasticsearch/search/SearchService.java](https://github.com/elasticsearch/elasticsearch/blob/f09ad507a44367ed6ca29c6f3dae7659a2da1994/src/main/java/org/elasticsearch/search/SearchService.java)),
> so no reason not to do as you suggest!
>
> (The existing elasticsearch-hadoop connector incidentally generates 1
> mapper per shard and scrolls through that, which might be as well as you
> can do anyway)
>
> On Monday, June 17, 2013 12:50:23 PM UTC-4, Oli wrote:
>
> > Hi Alex,
> >
> > I've recently been evaluating a very similar use case, you might be
> > interested in these threads (unique scroll ids[https://groups.google.com/forum/#!topic/elasticsearch/tqmwhuDADFw](https://groups.google.com/forum/#!topic/elasticsearch/tqmwhuDADFw)and characteristics
> > of scrolling[https://groups.google.com/forum/#!topic/elasticsearch/55ByVEr93bo](https://groups.google.com/forum/#!topic/elasticsearch/55ByVEr93bo)
> > ).
> >
> > I can't speak to different versions of ES or codebase specifics, but my
> > guess is that what you're seeing is similar to what I see in 0.20.4:
> >
> > - A \_scroll\_id will remain the same for some number of requests
> > issued against it, and at some point will change.
> > - Issuing multiple requests against the same \_scroll\_id will yield
> > subsequent results (at least for the overall resultset that that \_scroll\_id
> > represents before it changes), not the same results.
> > - You must use the most recent \_scroll\_id (as you mentioned) to be
> > sure to accurately get the full set of results for the query.
> >
> > If you're not seeing the scroll id change in 0.19. I would guess that
> > your query is small enough that it doesn't end up giving you a new id, but
> > that's just a hunch.
> >
> > To conclude, I don't \*think \*you'll be able to achieve what you want
> > using scrollers (at least that's the conclusion I've come to - I'd love to
> > hear otherwise). An alternative approach is using from/to attributes of a
> > normal query and have each mapper operate on a certain range of a query.
> >
> > - oli
> >
> > On Mon, Jun 17, 2013 at 9:28 AM, Alex at Ikanow [apig...@ikanow.com](mailto:apig...@ikanow.com)wrote:
> >
> > > I've been thinking about ways of running Hadoop jobs on the results of
> > > elasticsearch queries.
> > >
> > > I was hoping that I could just distribute a scrollid to the different
> > > mappers and then have them read and process batches of data served as
> > > needed from their closest es node (data locality isn't really an issue
> > > here, the hadoop and es nodes will normally not be co-located)
> > >
> > > The documentation is pretty clear that this shouldn't work (
> > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/scroll/)) "_Note_:
> > > the scroll\_id changes for each scroll request and only the most recent
> > > one should be used" ... but running on 0.19, the scroll id did appear
> > > to remain the same .... does anybody know which of the following is true:
> > >
> > > - On all versions the scrollid can change, though sometimes won't
> > > - It didn't change up until version XXX, but now does
> > > - It doesn't change, but we put that in because we might need to make
> > > it change in the future?
> > >
> > > (It would also require the get-more-data operation to be atomic across
> > > multiple nodes, which not be the case)
> > >
> > > Any pointers to good spots in the codebase to look at the inner workings
> > > of scrolls? (I always get a bit lost in the middle of the es code!)
> > >
> > > Thanks for any insight/pointers
> > >
> > > Alex
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "elasticsearch" group.
> > > To unsubscribe from this group and stop receiving emails from it, send
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:30am UTC](https://discuss.elastic.co/t/distributing-query-results-using-scrolling/12448/5 "2017-07-06T02:30:47Z")