# Reindex API does not complete the re-indexing

**URL:** <https://discuss.elastic.co/t/reindex-api-does-not-complete-the-re-indexing/366389>\
**Category:** Elasticsearch\
**Tags:** reindex\
**Created:** [September 11, 2024, 8:09am UTC](https://discuss.elastic.co/t/reindex-api-does-not-complete-the-re-indexing/366389 "2024-09-11T08:09:33Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![abhadauria](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhadauria/32/127899_2.png) [@abhadauria](https://discuss.elastic.co/u/abhadauria)\
**Post date:** [September 11, 2024, 8:09am UTC](https://discuss.elastic.co/t/reindex-api-does-not-complete-the-re-indexing/366389/1 "2024-09-11T08:09:33Z")

</div>

using ES version 7.17, after triggering the re-index API from query node, it stops re-indexing after some time and does not complete the re-indexing of all the docs in the source index, eg, we have ~110 million docs in the source index, but it stops re-indexing randomly after 20 million, if we trigger again after ~35 million and so on.

we are using automatic slicing in the re-index API, below is the sample request

curl -u user:pass -X POST "[https://localhost:9200/\_reindex?slices=100](https://localhost:9200/_reindex?slices=100)" -H "Content-Type: application/json" -d "{"source": {"index": "sourceIndex"},"dest": {"index": "targetIndex"}}" -k

is there some kind of timeout or anything else I am missing here, as we are not seeing any error on the console.

---

<div class="post-metadata">

**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\
**Post date:** [September 11, 2024, 4:20pm UTC](https://discuss.elastic.co/t/reindex-api-does-not-complete-the-re-indexing/366389/2 "2024-09-11T16:20:52Z")

</div>

Hi @abhadauria,

Looking at the [reindex API query parameters](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docs-reindex.html#docs-reindex-api-query-params) there is indeed a timeout operation, which default to 1 minute. So you could tweak that parameter.

Have you tried either [tweaking the number of slices as per the documentation guidance](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docs-reindex.html#docs-reindex-picking-slices) or perhaps tried the [automatic slicing option](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/docs-reindex.html#docs-reindex-automatic-slice)?

Let us know!

---

<div class="post-metadata">

**Author:** ![abhadauria](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhadauria/32/127899_2.png) [@abhadauria](https://discuss.elastic.co/u/abhadauria)\
**Post date:** [September 13, 2024, 5:45am UTC](https://discuss.elastic.co/t/reindex-api-does-not-complete-the-re-indexing/366389/3 "2024-09-13T05:45:15Z")

</div>

thanks for responding @carly.richmond, we have set number of slices equal to the number of shards in the index, i.e. 100

I will try setting up the timeout and see if it helps.

---

<div class="post-metadata">

**Author:** ![abhadauria](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhadauria/32/127899_2.png) [@abhadauria](https://discuss.elastic.co/u/abhadauria)\
**Post date:** [September 13, 2024, 10:34am UTC](https://discuss.elastic.co/t/reindex-api-does-not-complete-the-re-indexing/366389/4 "2024-09-13T10:34:03Z")

</div>

we are getting this error after increasing the timeout to 2h:

{"type":"es\_rejected\_execution\_exception","reason":"rejected execution of coordinating operation [coordinating\_and\_primary\_bytes=3319047037, replica\_bytes=0, all\_bytes=3319047037, coordinating\_operation\_bytes=76231083, max\_coordinating\_and\_primary\_bytes=3328599654]"}

we have allocated 30GB heap to the JVM where Elasticsearch is running.

size of the index we are trying to re-index is ~7 tb

---

<div class="post-metadata">

**Author:** ![carly.richmond](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/carly.richmond/32/104935_2.png) [@carly.richmond](https://discuss.elastic.co/u/carly.richmond)\
**Post date:** [September 13, 2024, 11:19am UTC](https://discuss.elastic.co/t/reindex-api-does-not-complete-the-re-indexing/366389/5 "2024-09-13T11:19:52Z")

</div>

Thanks for confirming. That's a pretty chunky index to reindex. Do your documents have any particular attribute that you could use to split them? That way you could try [specifying a query](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html) as part of the reindexing to split it using the [`query` option](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-api-request-body).

There is an option to [run the reindex asyncronously](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-task-api) as well that could help, but in your case I'm inclined to suggest trying that over smaller requests.

Hope that helps!