# How can i speed up getting all document in an index

**URL:** <https://discuss.elastic.co/t/how-can-i-speed-up-getting-all-document-in-an-index/236805>\
**Category:** Elasticsearch\
**Created:** [June 12, 2020, 2:01am UTC](https://discuss.elastic.co/t/how-can-i-speed-up-getting-all-document-in-an-index/236805 "2020-06-12T02:01:28Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![hyungsun\_lim](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hyungsun_lim/32/50790_2.png) [@hyungsun\_lim](https://discuss.elastic.co/u/hyungsun_lim)\
**Post date:** [June 12, 2020, 2:01am UTC](https://discuss.elastic.co/t/how-can-i-speed-up-getting-all-document-in-an-index/236805/1 "2020-06-12T02:01:28Z")

</div>

Hello, I have a question.

In my index, 50,000,000 documents in there.

I want to get all documents by python like below code

```auto
    import elasticsearch
    import json
    body_str = {"query": {"match_all": {}}}
    es_client = elasticsearch.Elasticsearch("address")
    doc = es_client.search(index = 'index',body = body_str, request_timeout=60, scroll='1m', size=1000)
    scroll_size = len(doc['hits']['hits'])
    sid = doc['_scroll_id']
    total_cnt = doc['hits']['total']
    while scroll_size > 0:
        doc = es_client.scroll(scroll_id=sid, scroll='10m',request_timeout=60 )
        print(doc)

```

but it takes too long time to get all documents.

I want to speed up.

The Network bandwidth python node to elasticsearch node is 1Gbit/sec

Can you give me an advice?

Thank you.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [June 12, 2020, 3:37am UTC](https://discuss.elastic.co/t/how-can-i-speed-up-getting-all-document-in-an-index/236805/2 "2020-06-12T03:37:45Z")

</div>

I'd recommend reading [Request body search | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#sliced-scroll)

And on the same page, let me show you this part:

> Scroll requests have optimizations that make them faster when the sort order is `_doc`. If you want to iterate over all documents regardless of the order, this is the most efficient option:
> 
> ```
> GET /_search?scroll=1m
> {
> "sort": [
> "_doc"
> ]
> }
> 
> ```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 10, 2020, 3:37am UTC](https://discuss.elastic.co/t/how-can-i-speed-up-getting-all-document-in-an-index/236805/3 "2020-07-10T03:37:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.