# Scroll Concept slows the search performance in latest Elastic version by php client **URL:** \ **Category:** Elasticsearch\ **Created:** [January 30, 2019, 3:22pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388 "2019-01-30T15:22:28Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![kshi\_yelpale000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kshi_yelpale000/32/42759_2.png) [@kshi\_yelpale000](https://discuss.elastic.co/u/kshi_yelpale000)\ **Post date:** [January 30, 2019, 3:22pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/1 "2019-01-30T15:22:28Z")
Hi all, I have migrated my application's elastic search version from 2.4 to 6.3, in which i saw decrease in performance figures. Size parameter in query request in old version was 1000000000, and by this i was getting 2719674 documents in just 42 seconds, but in new version we need to use scroll concept with size=10000, which gives me same result in 426 seconds which is really not acceptable. and code for that is as follows. ``` $client = $this->_createClient(); $aParams = $this->_setContentType($aParams); $docs = $client->search($aParams); $iScrollId = $docs['_scroll_id']; $aHits = $docs['hits']['hits']; $iScrollSize = $docs['hits']['total']; $aIDs = array(); if (is_array($aHits)) { do { foreach ($aHits as $aHit) { $iID = explode('_', $aHit['_id'])[0]; if ($this->_aSourceFields != null) { $aFieldValues = $aHit['_source']; $aIDs[$iID] = $aFieldValues; } else { array_push($aIDs, $iID); } if ($this->bHighlight) { $aHitHighlights[$iID] = $aHit['highlight']; } } $aScrollResults = array('scroll_id' => $iScrollId, 'scroll' => '1m'); $aEachScroll = $client->scroll($aScrollResults); $iScrollId = $aEachScroll['_scroll_id']; $aHits = $aEachScroll['hits']['hits']; $iScrollSize = count($aHits); } while ($iScrollSize > 0); } $client->clearScroll($this->_deleteScrollRequest($iScrollId)); ``` In the first request, scroll value is "10s"Please tell me if something to optimize the concept. Thanks and regards, Kshitij Yelpale ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 30, 2019, 3:53pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/2 "2019-01-30T15:53:13Z")
You can change `index.max_result_window` in your index settings to `1000000000` if you want to be able again to retrieve `1000000000` in one request. Not sure I'd do this though... Memory pressure wise. ---
**Author:** ![kshi\_yelpale000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kshi_yelpale000/32/42759_2.png) [@kshi\_yelpale000](https://discuss.elastic.co/u/kshi_yelpale000)\ **Post date:** [January 30, 2019, 3:56pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/3 "2019-01-30T15:56:07Z")
Hi David, Yes exactly that's why elastic give birth to scroll concept. Bye. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 30, 2019, 4:00pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/4 "2019-01-30T16:00:09Z")
No scroll was there from the begining. Anyway, if you were happy with `1000000000` and this is still working with your use case, then just use it again. ---
**Author:** ![kshi\_yelpale000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kshi_yelpale000/32/42759_2.png) [@kshi\_yelpale000](https://discuss.elastic.co/u/kshi_yelpale000)\ **Post date:** [January 30, 2019, 4:02pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/5 "2019-01-30T16:02:34Z")
Is there any other way ? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 30, 2019, 4:25pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/6 "2019-01-30T16:25:24Z")
I don't think there is. What is the problem? ---
**Author:** ![kshi\_yelpale000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kshi_yelpale000/32/42759_2.png) [@kshi\_yelpale000](https://discuss.elastic.co/u/kshi_yelpale000)\ **Post date:** [January 30, 2019, 4:52pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/7 "2019-01-30T16:52:28Z")
Searching time is more. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 30, 2019, 5:05pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/8 "2019-01-30T17:05:11Z")
With that solution? > [@dadoonet](#): > > You can change `index.max_result_window` in your index settings to `1000000000` if you want to be able again to retrieve `1000000000` in one request. Or with the "normal" scroll solution? ---
**Author:** ![kshi\_yelpale000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kshi_yelpale000/32/42759_2.png) [@kshi\_yelpale000](https://discuss.elastic.co/u/kshi_yelpale000)\ **Post date:** [January 30, 2019, 5:09pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/9 "2019-01-30T17:09:47Z")
if i use index.max\_result\_window = 1000000000 then it can cause memory issue (is there any way to avoid this), but in the scroll part can you please provide me solution which will boost the performance and give results as older version ? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 30, 2019, 5:34pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/10 "2019-01-30T17:34:48Z")
If you want the old behavior use index settings as I said and take the risk of memory pressure. If you want to have a safer option use scroll but with the price of more time to read all the content. Note that you can speed up extraction by using a sort on `_doc` as stated in documentation : [https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html) ---
**Author:** ![kshi\_yelpale000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kshi_yelpale000/32/42759_2.png) [@kshi\_yelpale000](https://discuss.elastic.co/u/kshi_yelpale000)\ **Post date:** [January 31, 2019, 4:58pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/11 "2019-01-31T16:58:00Z")
Hi David, Thank you, is it possible for you to tell me how sort by "\_doc" improves scroll performance, i mean some internal logic. ? Bye ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [January 31, 2019, 5:15pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/12 "2019-01-31T17:15:41Z")
It's because elasticsearch does not have to reorder documents while reading them. Just read them as they come. The default sort order on `_score` may have to reorder the documents which means that the deeper you go, the longer it takes. IIRC ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 28, 2019, 5:15pm UTC](https://discuss.elastic.co/t/scroll-concept-slows-the-search-performance-in-latest-elastic-version-by-php-client/166388/13 "2019-02-28T17:15:49Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.