# ElasticSearch indexes got corrupt after using Scroll API

**URL:** <https://discuss.elastic.co/t/elasticsearch-indexes-got-corrupt-after-using-scroll-api/129289>\
**Category:** Elasticsearch\
**Created:** [April 24, 2018, 10:42am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-got-corrupt-after-using-scroll-api/129289 "2018-04-24T10:42:08Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![nidhi](https://avatars.discourse-cdn.com/v4/letter/n/a5b964/32.png) [@nidhi](https://discuss.elastic.co/u/nidhi)\
**Post date:** [April 24, 2018, 10:42am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-got-corrupt-after-using-scroll-api/129289/1 "2018-04-24T10:42:08Z")

</div>

In my project requirement, I need to fetch more than 10k documents. I used ElasticSearch scroll api with python to do that. Here is my sample code -

url = '/\_search/scroll'  
scroll\_url='//\_search?scroll=2m'  
Query= {"query": {"bool": {"must": [{"match\_all": { }},{ "range": { "@timestamp": { "gt": "now-24h", "lt": "now-1h", "time\_zone": "-06:00" } } }],"must\_not": [],"should": []}},"from": 0,"size":10,"sort": [],"aggs": { }}  
response=requests.post(scroll\_url, json=query)  
sid = response['\_scroll\_id']  
hits=response['hits']  
total=hits["total"]  
while(total\>0):  
scroll = '2m'  
scroll\_query=json.dumps({"scroll" : scroll, "scroll\_id" : sid })  
response1=rquests.post(url,data=scroll\_query)  
# Update the scroll ID  
sid = response1['\_scroll\_id']  
# Get the number of results that we returned in the last scroll  
hits=response1['hits']  
total=len(response1['hits']['hits'])  
for each in hits['hits']:  
#iterate through my document

Scroll work perfect the way I wanted to, but later I was informed that because of this scroll elasticsearch schema got corrupted and it recreated the indexes.

Is it true that scroll modify the ES structure or something wrong with my code. Please let me know.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 22, 2018, 10:42am UTC](https://discuss.elastic.co/t/elasticsearch-indexes-got-corrupt-after-using-scroll-api/129289/2 "2018-05-22T10:42:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.