# Elasticsearch Reindex is slow compare to REALLOCATION

**URL:** https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785
**Category:** Elasticsearch
**Created:** [October 4, 2021, 6:30am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785 "2021-10-04T06:30:05Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![tusharnemade](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)
#### Post date: [October 4, 2021, 6:30am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/1 "2021-10-04T06:30:05Z")

</div>

Hello Team

I am using Elasticsearch version 7.8.0.

I have opened this thread for my understanding in regards to Elasticsearch internal operations.

A Cluster Version : 5.6.3.0  
B Cluster Version : 7.8.0.0

We are not using Complex Structures / MAPPINGS .. NO PIPLES / INJEST / DYNAMIC MAPPINGS.

We have mapping indexes already created in B Cluster and then performing re-indexing.

We are using TEXT , DATE , NUMBER Data types only.  
I have observed :

1. When I do Reindexing from A Cluster to B Cluster .. for Index of Size PRIMARY -5 SHARDS [12.5 GB] .. It takes 80-90 minutes.

Network is same / subnet is same .. only two different clusters...

1. If I do RE-ALLOCATION of Shards Within B Cluster  
OR  
I do increase replicas to 1 , from initial 0 during re-indexing ..

Its super fast ... less than 7-8 mins it completes..

Storage on both Cluster is SAME.

Could you please help me in its understanding..

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 4, 2021, 6:41am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/2 "2021-10-04T06:41:07Z")

</div>

Reindexing indexes all documents again and therfore rebuilds all data structures on disk and in memory. Reallocation on the other hand generally just involves moving aleready computed segment files between nodes which requires a lot less computational work and is a lot more efficient, but can not be done between clusters.

---

<div class="post-metadata">

### Author: ![tusharnemade](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)
#### Post date: [October 4, 2021, 6:45am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/3 "2021-10-04T06:45:18Z")

</div>

Okay ..

Thanks for the explanation @Christian_Dahlqvist

What can be done to achieve Faster Re-Indexing Between Cluster ..

What I did on target :

1. Made replica = 0
2. refresh = -1
3. translogs [flush threshold = 2GB] [durability as async]

Any thing else can be done to achieve speed ...

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 4, 2021, 6:53am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/4 "2021-10-04T06:53:16Z")

</div>

You can try slicing the reindex operation in order to improve parallelism.

---

<div class="post-metadata">

### Author: ![tusharnemade](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)
#### Post date: [October 4, 2021, 6:54am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/5 "2021-10-04T06:54:32Z")

</div>

SLICE Cannot be used with REMOTE Clusters ... during re-indexing operations ...

As per document I read ... this will give me error ...

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 4, 2021, 6:55am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/6 "2021-10-04T06:55:32Z")

</div>

Ah, good point. If you have large data volumes to move you could look into using snapshot and restore instead as that moves segments and do not require reprocessing.

---

<div class="post-metadata">

### Author: ![tusharnemade](https://avatars.discourse-cdn.com/v4/letter/t/67e7ee/32.png) [@tusharnemade](https://discuss.elastic.co/u/tusharnemade)
#### Post date: [October 4, 2021, 7:01am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/7 "2021-10-04T07:01:38Z")

</div>

Okay ...

Snapshots = We do not have ... due to our own limitations of INFRA.

So no other Settings at Index / Cluster Level which could boost my re-indexing ... ?

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [October 4, 2021, 7:19am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/8 "2021-10-04T07:19:27Z")

</div>

Not that I can think of. Just noticed that the difference in version would mean snapshot and restore would not work even if you had shared storage...

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 1, 2021, 7:19am UTC](https://discuss.elastic.co/t/elasticsearch-reindex-is-slow-compare-to-reallocation/285785/9 "2021-11-01T07:19:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
