# For exporting data shoud we use scroll or pit with search after?

**URL:** <https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475>\
**Category:** Elasticsearch\
**Created:** [September 15, 2022, 5:05am UTC](https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475 "2022-09-15T05:05:45Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Divit\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divit_sharma/32/32348_2.png) [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\
**Post date:** [September 15, 2022, 5:05am UTC](https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475/1 "2022-09-15T05:05:45Z")

</div>

For exporting data shoud we use scroll or pit with search after? Documentation says not to use scroll for deep pagination.

Assume that the data which we are querying would not change.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 15, 2022, 5:14am UTC](https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475/2 "2022-09-15T05:14:35Z")

</div>

I guess it will depend on which version you are using, but [for Elasticsearch 8.4 the recommendation is](https://www.elastic.co/guide/en/elasticsearch/reference/8.4/scroll-api.html):

> We no longer recommend using the scroll API for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the [`search_after`](https://www.elastic.co/guide/en/elasticsearch/reference/8.4/paginate-search-results.html#search-after) parameter with a point in time (PIT).

---

<div class="post-metadata">

**Author:** ![Divit\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divit_sharma/32/32348_2.png) [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\
**Post date:** [September 20, 2022, 7:17pm UTC](https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475/3 "2022-09-20T19:17:22Z")

</div>

@dadoonet Sorry for tagging you but I just wanted to confirm this. I found your reply to below saying that `_scroll` is preferred over `search_after` for pulling whole dataset. Does it still apply ?  
The documentation has not mentioned this clearly.

> [@Recommendation to use search after instead of scrolling](https://discuss.elastic.co/t/recommendation-to-use-search-after-instead-of-scrolling/268293):
>
> Hello everybody If you want to read an index completely, you can read in the documentation that the search after function should be used from 10,000 documents (instead of scroll). We have now tested this and unfortunately we have noticed a slowdown by a factor of 10. we sort with the \_id meta field. If we use a technical id as a number for sorting, the speed is the same as when scrolling. unfortunately we do not have a technical number field in our use case. why is sorting with a text field …

It still makes sense to me to use `_scroll` since for `search_after` I still need to move through the hits to find the sort value in the last document whereas in `_scroll` I just use the `scroll_id` for the next batch.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [September 21, 2022, 6:41am UTC](https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475/4 "2022-09-21T06:41:50Z")

</div>

What @Christian_Dahlqvist said is 100% correct.

I should edit my old answer I guess. 😉

---

<div class="post-metadata">

**Author:** ![Divit\_Sharma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/divit_sharma/32/32348_2.png) [@Divit\_Sharma](https://discuss.elastic.co/u/Divit_Sharma)\
**Post date:** [September 21, 2022, 7:13am UTC](https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475/5 "2022-09-21T07:13:32Z")

</div>

Thank you @dadoonet

I also had another query about `_scroll`

> [@Sliced scroll returning more hits than normal search (without slice)](https://discuss.elastic.co/t/sliced-scroll-returning-more-hits-than-normal-search-without-slice/314599):
>
> In short, normal search hits \< scroll slice 1 hits + scroll slice 2 hits When I add up sliced scroll hits, it is more than total no of documents returned from single search. Normal search query GET index\*/\_search { "track\_total\_hits": true, "sort": [{ "@timestamp": { "order": "asc", "unmapped\_type": "boolean" } }], "\_source": false, "query": { "bool": { "must": [], "filter": [ { "range": { "@timesta…

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 19, 2022, 7:14am UTC](https://discuss.elastic.co/t/for-exporting-data-shoud-we-use-scroll-or-pit-with-search-after/314475/6 "2022-10-19T07:14:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.