# Search & Scroll using python

**URL:** https://discuss.elastic.co/t/search-scroll-using-python/154841
**Category:** Elasticsearch
**Created:** [October 31, 2018, 12:34pm UTC](https://discuss.elastic.co/t/search-scroll-using-python/154841 "2018-10-31T12:34:25Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Moshe\_Hayun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/moshe_hayun/32/37150_2.png) [@Moshe\_Hayun](https://discuss.elastic.co/u/Moshe_Hayun)
#### Post date: [October 31, 2018, 12:34pm UTC](https://discuss.elastic.co/t/search-scroll-using-python/154841/1 "2018-10-31T12:34:25Z")

</div>

Hi,

I have an issue where I am trying to use the search & scroll mechanism to query large amount of information.  
[https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search](https://elasticsearch-py.readthedocs.io/en/master/api.html#elasticsearch.Elasticsearch.search)

From time to time due to connectivity issues I am getting IncompleteRead (expected to read \<x\> more bytes) followed by MaxRetryError.

Using tcpdump I've noticed that I've received partial response json when this occurs.  
I can't avoid the connectivity issues in my organization.

My problem is that when this error occurs, the scroll (from the elastic api) doesn't repeat the same request.  
It moves on to the next request and I am losing some of the data.

E.g. I am querying 1 million records in chunks of 1000.  
If I had 100 connectivity issues during the period it took, I will receive 900k records only.

I don't want to implement the same thing on my own with retries on the same request.  
Is there a way using the api to retry the same request in case it fails?

Thanks a lot.

---

<div class="post-metadata">

### Author: ![Moshe\_Hayun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/moshe_hayun/32/37150_2.png) [@Moshe\_Hayun](https://discuss.elastic.co/u/Moshe_Hayun)
#### Post date: [November 5, 2018, 12:05pm UTC](https://discuss.elastic.co/t/search-scroll-using-python/154841/2 "2018-11-05T12:05:00Z")

</div>

Checking again if someone is familiar

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 3, 2018, 12:05pm UTC](https://discuss.elastic.co/t/search-scroll-using-python/154841/3 "2018-12-03T12:05:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
