# Search\_after vs deep pagination **URL:** \ **Category:** Elasticsearch\ **Created:** [May 22, 2017, 8:37pm UTC](https://discuss.elastic.co/t/search-after-vs-deep-pagination/86738 "2017-05-22T20:37:26Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [May 22, 2017, 8:37pm UTC](https://discuss.elastic.co/t/search-after-vs-deep-pagination/86738/1 "2017-05-22T20:37:26Z")
[Documentation](https://www.elastic.co/guide/en/elasticsearch/reference/master/search-request-search-after.html) suggests that search\_after is suggested compared to deep pagination, but doesn't seem to explain why. Or at least I didn't understand the details. Could someone explain why we should use search\_after vs deep pagination? ---
**Author:** ![KodrAus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kodraus/32/22798_2.png) [@KodrAus](https://discuss.elastic.co/u/KodrAus)\ **Post date:** [May 22, 2017, 10:43pm UTC](https://discuss.elastic.co/t/search-after-vs-deep-pagination/86738/2 "2017-05-22T22:43:49Z")
So as I understand it, the issue with deep pagination using `from` + `size` is that Elasticsearch will load all documents into memory up to the page you ask for, then sort them and return the slice you asked for. This is the result window. The memory needed per request is proportional to **`from` + `size`** , not just `size`. So it's not good for paging through large result sets because the more results you skip, the slower your query becomes. Scroll/search after let you do deep scrolling where you only fetch the next result set in an efficient manner, but can't jump between random pages. It solves the scaling problem of `from` + `size`. Scroll needs to maintain a context per scroll though, which isn't great for lots of users doing lots of scrolling. Search after is like scroll, but stateless. So there isn't any additional state stored on the Elasticsearch end, so you don't have the scroll context scaling issue and you don't have the `from` + `size` scaling issue. It does share the constraint that you can only page forwards. Usually I work around the random paging issue through user experience. Most people don't usually want to randomly page through results. They search, scroll a bit, filter, scroll a bit more, rinse-and-repeat. If scaling isn't a problem for you though then `from` + `size` is a really simple solution to paging. ---
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [May 22, 2017, 11:05pm UTC](https://discuss.elastic.co/t/search-after-vs-deep-pagination/86738/3 "2017-05-22T23:05:36Z")
Thank you for the response. This basically means search\_after and scroll are exactly the same except the fact that search\_after is stateless. If the query includes paginating to page 500 with a page size of 100 along with sort on a couple of fields and a few filters, would ES still sort and load all documents to memory and only return the requested size? Scaling definitely is a problem. Hitting a page post 5.4M leads to OOM (I tried increasing the default 10k to 15M for testing purpose). ---
**Author:** ![KodrAus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kodraus/32/22798_2.png) [@KodrAus](https://discuss.elastic.co/u/KodrAus)\ **Post date:** [May 23, 2017, 1:13am UTC](https://discuss.elastic.co/t/search-after-vs-deep-pagination/86738/4 "2017-05-23T01:13:53Z")
> [@animageofmine](#): > > This basically means search\_after and scroll are exactly the same except the fact that search\_after is stateless. There's probably more subtlety to it than that, someone else who knows more might come along and shed some more light on it 🙂 As far as the end-user is concerned though I think it's safe to say they operate in the same way. > [@animageofmine](#): > > If the query includes paginating to page 500 with a page size of 100 along with sort on a couple of fields and a few filters, would ES still sort and load all documents to memory and only return the requested size? That's right, it will run your filters to determine which documents match, collect them, sort them and then only return the documents after `from`. Which as you pointed out doesn't scale so well. That's why the limit is 10,000 so you get an error suggesting you look for alternative solutions than OOM, which is bad news for your node. ---
**Author:** ![animageofmine](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animageofmine](https://discuss.elastic.co/u/animageofmine)\ **Post date:** [May 23, 2017, 3:00pm UTC](https://discuss.elastic.co/t/search-after-vs-deep-pagination/86738/5 "2017-05-23T15:00:40Z")
> [@KodrAus](#): > > That's right, it will run your filters to determine which documents match, collect them, sort them and then only return the documents after from. Which as you pointed out doesn't scale so well. That's why the limit is 10,000 so you get an error suggesting you look for alternative solutions than OOM, which is bad news for your node. So, just to confirm, if ES runs filters and sorts the documents with a request for a document after 5M, wouldn't it possibly lead to OOM? I am just looking for a good solution for deep pagination with filters and sorts included in the query. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 20, 2017, 3:01pm UTC](https://discuss.elastic.co/t/search-after-vs-deep-pagination/86738/6 "2017-06-20T15:01:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.