# Reindex API not distributing load when using slicing

**URL:** <https://discuss.elastic.co/t/reindex-api-not-distributing-load-when-using-slicing/325770>\
**Category:** Elasticsearch\
**Tags:** reindex\
**Created:** [February 16, 2023, 7:20pm UTC](https://discuss.elastic.co/t/reindex-api-not-distributing-load-when-using-slicing/325770 "2023-02-16T19:20:59Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmench](https://avatars.discourse-cdn.com/v4/letter/j/838e76/32.png) [@jmench](https://discuss.elastic.co/u/jmench)\
**Post date:** [February 16, 2023, 7:20pm UTC](https://discuss.elastic.co/t/reindex-api-not-distributing-load-when-using-slicing/325770/1 "2023-02-16T19:20:59Z")

</div>

I have an index with 3 nodes running, each node having 1 primary and 1 replica shard:

```auto
index shard prirep state node
source-index 0 p STARTED eck-elasticsearch-es-default-2
source-index 0 r STARTED eck-elasticsearch-es-default-1
source-index 1 p STARTED eck-elasticsearch-es-default-0
source-index 1 r STARTED eck-elasticsearch-es-default-1
source-index 2 p STARTED eck-elasticsearch-es-default-0
source-index 2 r STARTED eck-elasticsearch-es-default-2

```

I have a need to use the reindex API to remove some fields from the existing documents. I create the new index with some recommended settings to increase indexing speed, as such:

```auto
put /destination-index
{
  "mappings": {
    "dynamic": "strict",
    "properties": {...}
  },
  "settings": {
    "number_of_shards": 3,
    "number_of_replicas": 0,
    "refresh_interval": -1
  }
}

```

I run the reindex request as follows:

```auto
POST _reindex?wait_for_completion=false&slices=auto
{
  "source": {
    "index": "source-index",
    "_source": "${varList}"
  },
  "dest": {
    "index": "destination-index"
  }
}

```

When I run this request, it creates 3 slices to run in parallel. This follows with the docs from [Automatic Slicing](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-automatic-slice) that states: “This setting (`slices=auto`) will use one slice per shard, up to a certain limit”. Given the 3 primary shards as shown above, this all makes complete sense.

What doesn't make sense is that I check the child tasks of the main `reindex` action, and each child task with the action `indices:data/write/reindex` is using the same node:

```auto
action parent_task_id type running_time node
indices:data/write/reindex l7bY43OiRGquG6KcBAa4Jw:327586044 transport 15.3m eck-elasticsearch-es-default-0
indices:data/write/reindex l7bY43OiRGquG6KcBAa4Jw:327586044 transport 15.3m eck-elasticsearch-es-default-0
indices:data/write/reindex l7bY43OiRGquG6KcBAa4Jw:327586044 transport 15.3m eck-elasticsearch-es-default-0

```

It feels like this is unexpected given the wording of the documentation for using slices with `_reindex`. What I would expect is that the `reindex` action would utilize all 3 nodes, each utilizing a thread per primary shard, to write to another primary shard in the destination index. Something like:

```auto
source primary shard 1 on node A reindexing to destination primary shard 1 on node A
source primary shard 2 on node B reindexing to destination primary shard 2 on node B
etc...

```

I bring this up because the reindexing process feels slow and unpredictable with each test run that I do. Just curious about utilizing one node for all 3 slices vs distributing the reindexing to every node and spreading the load. Having all 3 slices being processed by the same node at the same time is much slower than a reindex process with the same number of documents without slices.

---

<div class="post-metadata">

**Author:** ![jmench](https://avatars.discourse-cdn.com/v4/letter/j/838e76/32.png) [@jmench](https://discuss.elastic.co/u/jmench)\
**Post date:** [February 22, 2023, 3:37pm UTC](https://discuss.elastic.co/t/reindex-api-not-distributing-load-when-using-slicing/325770/2 "2023-02-22T15:37:24Z")

</div>

Any information on this topic would be helpful. Thanks!

---

<div class="post-metadata">

**Author:** ![jmench](https://avatars.discourse-cdn.com/v4/letter/j/838e76/32.png) [@jmench](https://discuss.elastic.co/u/jmench)\
**Post date:** [March 9, 2023, 3:02pm UTC](https://discuss.elastic.co/t/reindex-api-not-distributing-load-when-using-slicing/325770/3 "2023-03-09T15:02:08Z")

</div>

Still curious about this!

---

<div class="post-metadata">

**Author:** ![jmench](https://avatars.discourse-cdn.com/v4/letter/j/838e76/32.png) [@jmench](https://discuss.elastic.co/u/jmench)\
**Post date:** [March 16, 2023, 5:25pm UTC](https://discuss.elastic.co/t/reindex-api-not-distributing-load-when-using-slicing/325770/4 "2023-03-16T17:25:29Z")

</div>

Still curious!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 13, 2023, 5:25pm UTC](https://discuss.elastic.co/t/reindex-api-not-distributing-load-when-using-slicing/325770/5 "2023-04-13T17:25:48Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.