I failed to mention one of the reasons for preferring searching over scrolling. When scrolling, you effectively have something akin to a cursor over a Lucene segment during the life of the scroll. While this resource is in use, ES cannot allow that segment to be merged and reclaimed per the normal segment lifecycle. There is no such open resource in the search case.
I used scroll not scan with uri search
Elasticsearch Platform — Find real-time answers at scale | Elastic
On May 21, 2014 1:04 PM, "joergprante@gmail.com" joergprante@gmail.com wrote:
I wonder how you did that, the scan search type is defined by doing no sort and read directly from the shard segments. From the docs:
"The costly part of deep pagination is the global sorting of results, but if we disable sorting then we can return all documents quite cheaply. To do this, we use the scan search type. Scan instructs Elasticsearch to do no sorting, but to just return the next batch of results from every shard which still has results to return."
Elasticsearch Platform — Find real-time answers at scale | Elastic
Jörg
On Wed, May 21, 2014 at 9:03 PM, Nikita Tovstoles nikita.tovstoles@gmail.com wrote:
I was able to sort by property while using scroll
On May 21, 2014 11:57 AM, "joergprante@gmail.com" joergprante@gmail.com wrote:
You described a browsing scenario to present products to users. Most users want to sort the products, for e.g. by price, brands, popularity, or actuality. With scan/scroll, you can not sort.
Jörg
On Wed, May 21, 2014 at 7:32 PM, Nikita Tovstoles nikita.tovstoles@gmail.com wrote:
Valid point. However, there does appear to exist a (slight) perf penalty for retrieving 10,000th item vs. 1st for the same query (ie from=0 vs from=10000) - about 10x REST latency in my manual tests.
I imagine no such penalty exists with scroll. If so the trade off between using scroll v. pagination appears to be latency vs flexibility. While you're probably right wrt pagination being the better approach in 90% of use cases, in some cases scroll (ex: infinite scroll) appears a better fit.
So, aside from lack of flexibility and higher memory demands, are there other reasons not to use scroll in real time?
On Wed, May 21, 2014 at 10:17 AM, Andrew Selden andrew.selden@elasticsearch.com wrote:
Nikita,
Pagination is usually implemented using a search request with the 'from' and 'size' parameters [1]. Technically you can do pagination with scroll, as you have seen. However, sooner or later you will want to get more sophisticated in your logic and that will inevitably include sorting and faceting/aggregating, which the scroll API does not support.
Better to start off on a sound footing by using search.
Andrew
- Elasticsearch Platform — Find real-time answers at scale | Elastic
On May 21, 2014, at 9:09 AM, Nikita Tovstoles nikita.tovstoles@gmail.com wrote:
I am looking at scroll to support a use case of paging through a list of products resulting from user checking off a number of product attribute filters in a typical ecommerce 'search & browse' scenario. Think someone wanting to page through a sub-set of diapers of specific brands.
scroll seems like a natural fit there, performing well enough against our small dataset (master product list on the order of 100K products; user selections paring down the result set to < 10K) but the docs say not to use it, seemingly:
"Scrolling is not intended for real time user requests, it is intended for cases like scrolling over large portions of data that exists within elasticsearch to reindex it for example."
Does the above warning presume much larger query result sets (and resulting memory allocation requirements on the ES server)? Or is there another reason?
thanks,
-nikita
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8ae9c470-a34a-46da-90fb-409fdfc08e71%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/kuyLud4gPrk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3859CD3A-BD7C-47FF-A45C-1AFC816E86B4%40elasticsearch.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJwaA21SSgMgn0ee6bo%3DDck5xzFmwnNvpi7YxS3FbkPjXDGK-g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/kuyLud4gPrk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEXE%3DhifqOW5a4JfLxJrsswGr4k4coJ7pa%2BoGZ9W753sQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJwaA20WKTpcoXRHvNAQJ7F5mremX6pkM-bH7KnnV%3D_BPwxwLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/kuyLud4gPrk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoENUSBmmR%3D7G1M9WpjDqcnUVwaqTnKoYJ%2Bbq8NPOJXtjw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAJwaA23h6fqFwWtqcgXO2n%3DUyf8p-ASg24vjM_MFSwJTKg9QKw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.