# Slow reindex operation on heavy index

**URL:** https://discuss.elastic.co/t/slow-reindex-operation-on-heavy-index/198320
**Category:** Elasticsearch
**Created:** [September 5, 2019, 10:26pm UTC](https://discuss.elastic.co/t/slow-reindex-operation-on-heavy-index/198320 "2019-09-05T22:26:47Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![viniciof](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/viniciof/32/53454_2.png) [@viniciof](https://discuss.elastic.co/u/viniciof)
#### Post date: [September 5, 2019, 10:26pm UTC](https://discuss.elastic.co/t/slow-reindex-operation-on-heavy-index/198320/1 "2019-09-05T22:26:47Z")

</div>

Hi all,

How can I speed up a long running reindex operation ?

This is being done from a source index of around 4.2TB with 16 shards each of around 300GB~ in a 10 data nodes clusters.

Target index is 90 shards. I've set # of replicas to 0 and refresh rate to -1 to try to speed things up. BUT at this point it has only indexed 1GB in the last 3 hours, which is very slow.

Here the monitoring statistics:

 ![compilation%20monitoring](https://us1.discourse-cdn.com/elastic/original/3X/b/6/b6fea2c905e39da426df381979a5b646081b3b2e.png)

What else can be done to speed this up ?

Regards,

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [September 7, 2019, 10:41am UTC](https://discuss.elastic.co/t/slow-reindex-operation-on-heavy-index/198320/2 "2019-09-07T10:41:58Z")

</div>

Have you tried [slicing the reindex operation](https://www.elastic.co/guide/en/elasticsearch/reference/7.3/docs-reindex.html#docs-reindex-slice)?

---

<div class="post-metadata">

### Author: ![viniciof](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/viniciof/32/53454_2.png) [@viniciof](https://discuss.elastic.co/u/viniciof)
#### Post date: [September 7, 2019, 9:05pm UTC](https://discuss.elastic.co/t/slow-reindex-operation-on-heavy-index/198320/3 "2019-09-07T21:05:18Z")

</div>

This is the solution. Thanks! It helped indeed. I also merged the source index in a single segment as I don't expect any further writes to it anytime soon. Also disabled all type of shard allocation throughout the cluster and now my reindex is avg ~15,000 docs/sec which is the best historical indexing rate I've ever had in this cluster 🙂

```
POST _reindex?wait_for_completion=false&slices=20&refresh
{
  "source": {
    "index": "puma.compilation.pipeline.96f19f5b-bc84-4d4b-8694-b80a293e78e4-latest",
    "size": 500,
    "query": {
"range": {
      "ibi_logtime": {
        "gte": "now-9M/M"
      }
    }
    }
  },
  "dest": {
    "index": "puma.compilation.pipeline.96f19f5b-bc84-4d4b-8694-b80a293e78e4-optimized"
  }
}
```

---

<div class="post-metadata">

### Author: ![viniciof](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/viniciof/32/53454_2.png) [@viniciof](https://discuss.elastic.co/u/viniciof)
#### Post date: [September 8, 2019, 2:02am UTC](https://discuss.elastic.co/t/slow-reindex-operation-on-heavy-index/198320/4 "2019-09-08T02:02:05Z")

</div>

A disclaimer here, since my reindex operations take too long, I would not recommend anybody to disable allocations at cluster level if there new indices being created in the cluster (it would cause red state)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 6, 2019, 2:02am UTC](https://discuss.elastic.co/t/slow-reindex-operation-on-heavy-index/198320/5 "2019-10-06T02:02:05Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.