# How to optimize a reindex operation to perform really fast on big source index

**URL:** https://discuss.elastic.co/t/how-to-optimize-a-reindex-operation-to-perform-really-fast-on-big-source-index/198721
**Category:** Elasticsearch
**Created:** [September 9, 2019, 3:11pm UTC](https://discuss.elastic.co/t/how-to-optimize-a-reindex-operation-to-perform-really-fast-on-big-source-index/198721 "2019-09-09T15:11:01Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![viniciof](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/viniciof/32/53454_2.png) [@viniciof](https://discuss.elastic.co/u/viniciof)
#### Post date: [September 9, 2019, 3:11pm UTC](https://discuss.elastic.co/t/how-to-optimize-a-reindex-operation-to-perform-really-fast-on-big-source-index/198721/1 "2019-09-09T15:11:01Z")

</div>

Hi everyone,

I need to reindex a 4.1TB index that was created with very little amount of shards originally (no capacity planning & growth forecast was done)

Nothing is being written currently in the source index and it's currently forced merged into a single segment for better "read/search" performance.

Target index is created from scratch (empty) and both refresh interval & replicas are disabled

I'm using slicing in the reindex tasks, one per shard in the source index (primary & replicas - is this ok ? should it be done only for primary shards ? )

Also batch size is 1000 records (when I set it to 2000 it blows up my cluster the whole thing starts giving error code 500 )

Is there something else that can be done here to make it run faster ? Also, is there a way to use scrolling (specify longer scroll TTL time in the body of reindex for search context )

```
POST _reindex?wait_for_completion=false&slices=20
{
  "source": {
    "index": "puma.compilation.pipeline.96f19f5b-bc84-4d4b-8694-b80a293e78e4-latest",
    "size": 1000,
    "query": {
"range": {
      "ibi_logtime": {
        "gte": "now-9M/M"
      }
    }
    }
  },
  "dest": {
    "index": "puma.compilation.pipeline.96f19f5b-bc84-4d4b-8694-b80a293e78e4-optimized"
  }
}

```

Appreciate any feedback that can be provided on this,

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 7, 2019, 3:11pm UTC](https://discuss.elastic.co/t/how-to-optimize-a-reindex-operation-to-perform-really-fast-on-big-source-index/198721/2 "2019-10-07T15:11:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.