# Sorted scroll internal workings for performance optimizations **URL:** \ **Category:** Elasticsearch\ **Created:** [June 17, 2015, 6:51pm UTC](https://discuss.elastic.co/t/sorted-scroll-internal-workings-for-performance-optimizations/23866 "2015-06-17T18:51:51Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![lisak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lisak/32/618_2.png) [@lisak](https://discuss.elastic.co/u/lisak)\ **Post date:** [June 17, 2015, 6:51pm UTC](https://discuss.elastic.co/t/sorted-scroll-internal-workings-for-performance-optimizations/23866/1 "2015-06-17T18:51:51Z")
Hi, could please anybody confirm my hypothesis regarding sorted scroll? 1. I assume that normal sorted search : - loads all fields we sort on into memory in case of fieldData, in case of `doc_values` we are just not limited by heap (but hard to imagine sorting without using memory 🙂 - returns all hits sorted 1. I can't really imagine how sorted scrolling is implemented though because you kinda have to sort say billion documents over a `doc_values` field, at least partially. Until then scrolling cannot be started, because you wouldn't know what really is the first document to fetch, right? BUT scrolling starts returning sorted results immediately although very slowly (800 docs/s) which is 10 times slower than scanning the same thing. - what is the bottleneck here? I have like 7% IO wait, which eliminates issue with `doc_values`, low network traffic but CPU is at 98% which seems like there was a sorting thread at the background providing scrolling/searching thread with documents to retrieve. If it is like this, is this sorting thread a bottleneck? Meaning that reading 300M doc\_values fields and sorting them is IO & computational heavy operation that makes it 10 times slower ? - Is there any way to improve it? Since all fields in my documents are `doc_values` and it is search/scroll only node. I assume that I should split memory available to something like 30%-heap/70%-off-heap, right? Providing sufficient amount of OS ram for FS cache. Is there anything else I'm not aware of? ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [June 18, 2015, 12:23pm UTC](https://discuss.elastic.co/t/sorted-scroll-internal-workings-for-performance-optimizations/23866/2 "2015-06-18T12:23:19Z")
Hi, The way sorted scroll works is that elasticsearch remembers the last returned document on every shard. Then when you ask for the next page, all documents that match your query and compare better than this last returned document are ignored. This explains why sorted scroll is less efficient than scan scroll, since scan can just go back to where it had stopped before, collect ${page\_size} documents and return. In contrast, sorted scroll needs to visit all matching documents for every page. > [@lisak](#): > > but CPU is at 98% This part is interesting, can you try to capture the [hot threads](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html) output and share it? This could give interesting information about where your CPU is spending time. ---
**Author:** ![lisak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lisak/32/618_2.png) [@lisak](https://discuss.elastic.co/u/lisak)\ **Post date:** [June 19, 2015, 11:01am UTC](https://discuss.elastic.co/t/sorted-scroll-internal-workings-for-performance-optimizations/23866/3 "2015-06-19T11:01:21Z")
Thank you @jpountz, before I run it again (I have a different 10hour job running now), could you please give me some optimization pointers regarding sorted scrolling with `doc_values` only? I guess that : 1. FieldData cache is irrelevant here because of the `doc_values` 2. filter cache is still important, but I don't think that it needs more than 10%, or does it? 3. Is `30%-heap/70%-off-heap` ratio optimal for search-only node with `doc_values` fields only? Since everything is in PageCache/Fs cache instead of heap ---
**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\ **Post date:** [June 19, 2015, 11:29am UTC](https://discuss.elastic.co/t/sorted-scroll-internal-workings-for-performance-optimizations/23866/4 "2015-06-19T11:29:43Z")
Indeed the fielddata cache is mostly irrelevant if you have doc values, unless you run eg. terms aggregations which need to load a global ordinal map on top of doc values and store it in the fielddata cache. 10% is indeed already a lot for the filter cache. If you want to optimize its usage, a better idea to increasing its size is usually to explicitly disable caching on filters which are not reused -- by the way Elasticsearch 2.0 will cache filters much less aggressively for that reason. If your node can handle the load at a 30/70 ratio, then it sounds great. In general, the more you can give to the filesystem cache, the better. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:06am UTC](https://discuss.elastic.co/t/sorted-scroll-internal-workings-for-performance-optimizations/23866/5 "2017-07-06T00:06:28Z")