# Elasticsearch Reindex Big Index

**URL:** <https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569>\
**Category:** Elasticsearch\
**Created:** [November 14, 2017, 3:33pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569 "2017-11-14T15:33:18Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![eng.behjati](https://avatars.discourse-cdn.com/v4/letter/e/6bbea6/32.png) [@eng.behjati](https://discuss.elastic.co/u/eng.behjati)\
**Post date:** [November 14, 2017, 3:33pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569/1 "2017-11-14T15:33:18Z")

</div>

Hi everyone!

I have a problem with elasticsearch reindex API.  
when I run reindex API on big index , after ES reindexes certain number of documents, it responses "search context not found" exception. I should mention that I use python client.  
I know this arise because scroll time expire but I could not find any way to set scroll in reindex API parameters to avoid this.  
what should I do?

#reindex

---

<div class="post-metadata">

**Author:** ![jpountz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jpountz/32/45836_2.png) [@jpountz](https://discuss.elastic.co/u/jpountz)\
**Post date:** [November 21, 2017, 1:43pm UTC](https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569/2 "2017-11-21T13:43:27Z")

</div>

Even on a large index, this should not occur. What does the reindex call look like?

---

<div class="post-metadata">

**Author:** ![worroc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/worroc/32/27176_2.png) [@worroc](https://discuss.elastic.co/u/worroc)\
**Post date:** [November 26, 2017, 11:59am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569/3 "2017-11-26T11:59:36Z")

</div>

Hi All,  
Have the same issue. I reindex index with 23033921 documents but new index contains only 10925105.  
I have run ~15 tasks 10 slices each.  
in total it should reindex 222701787 documents but only 153689150 were actualy copied.  
69,012,637 is missing.

How can we fix it?

Here is a task log,

samples  
{  
"slice\_id": 5,  
"total": 1896711,  
"updated": 0,  
"created": 30000,  
"deleted": 0,  
"batches": 3,  
"version\_conflicts": 0,  
"noops": 0,  
"retries": {  
"bulk": 0,  
"search": 0  
},  
"throttled\_millis": 0,  
"requests\_per\_second": -1,  
"throttled\_until\_millis": 0  
},

error  
{  
"type": "search\_phase\_execution\_exception",  
"reason": "all shards failed",  
"phase": "query",  
"grouped": true,  
"failed\_shards": [  
{  
"shard": -1,  
"index": null,  
"reason": {  
"type": "search\_context\_missing\_exception",  
"reason": "No search context found for id [69020]"  
}  
},

"error": {  
"type": "search\_phase\_execution\_exception",  
"reason": "all shards failed",  
"phase": "query",  
"grouped": true,  
"failed\_shards": [  
{  
"shard": -1,  
"index": null,  
"reason": {  
"type": "search\_context\_missing\_exception",  
"reason": "No search context found for id [69021]"  
}  
},

#reindex

---

<div class="post-metadata">

**Author:** ![worroc](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/worroc/32/27176_2.png) [@worroc](https://discuss.elastic.co/u/worroc)\
**Post date:** [November 27, 2017, 7:01am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569/4 "2017-11-27T07:01:54Z")

</div>

the elastic version is 5.6.3

---

<div class="post-metadata">

**Author:** ![eng.behjati](https://avatars.discourse-cdn.com/v4/letter/e/6bbea6/32.png) [@eng.behjati](https://discuss.elastic.co/u/eng.behjati)\
**Post date:** [December 18, 2017, 8:41am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569/5 "2017-12-18T08:41:24Z")

</div>

Hi jpountz!  
thanks for your attention.

I used python client.  
first I used directly reindex api and that was look like this:  
elastic\_client.reindex(  
{  
"source": {  
"index": old\_index\_name  
},  
"dest": {  
"index": new\_index\_name  
}  
})  
it didn't work. so I changed it to this and it's looks like this :

helpers.reindex(elastic\_client,  
source\_index=old\_index\_name,  
target\_index=new\_index\_name, query={"query": {  
"match\_all": {} }},  
chunk\_size = 1000,  
scroll = "1h")

but still I get that error that says "No search context found for id [69021]" after almost half of index reindexed.

also my elasticsearch version is 5.5.1

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 15, 2018, 8:41am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-big-index/107569/6 "2018-01-15T08:41:30Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.