# Is reindex API works in parallel by shard? **URL:** \ **Category:** Elasticsearch\ **Created:** [December 6, 2016, 9:02am UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142 "2016-12-06T09:02:21Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Post date:** [December 6, 2016, 9:02am UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142/1 "2016-12-06T09:02:21Z")
I use Hadoop plugin a lot, it's quite fast because it takes advantage of shards to work in parallel. Is reindex API do the same? (or can) ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [December 6, 2016, 1:13pm UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142/2 "2016-12-06T13:13:04Z")
Hi @ebuildy, I am not sure whether I've misunderstood you. The reindex API basically uses sliced scrolls to retrieve documents from the source index and uses the bulk API to put the documents into the destination index. The number of shards is determined by the index settings and not really related to the reindex API. Daniel ---
**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Post date:** [December 6, 2016, 1:30pm UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142/3 "2016-12-06T13:30:57Z")
I am wondering if reindex API works like es4Hadoop does: scrolling in parrallel from each shard ====\> \_bulk Let's say you have 3 data nodes, you want to reindex an index with 3 shards, that mean, 1 data node could copy 1 shard, you see? ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [December 6, 2016, 1:44pm UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142/4 "2016-12-06T13:44:30Z")
Hi @ebuildy, ok, got you! 🙂 The reindex action is coordinated by one node in the cluster. Processing of the sliced scrolls is done in parallel however. Daniel ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [December 6, 2016, 6:13pm UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142/5 "2016-12-06T18:13:48Z")
> [@danielmitterdorfer](#): > > The reindex action is coordinated by one node in the cluster. Processing of the sliced scrolls is done in parallel however. In 5.1 where is it implemented at all. Before that reindex was always a single process. That isn't to say it is a single thread, just that it isn't forked. The search and bulk stages ran on as many threads as they usually do in Elasticsearch. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [December 7, 2016, 7:02am UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142/6 "2016-12-07T07:02:14Z")
Thanks for the additional detail on my answer @nik9000. I was looking at the implementation on the master branch indeed. Daniel ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 4, 2017, 7:02am UTC](https://discuss.elastic.co/t/is-reindex-api-works-in-parallel-by-shard/68142/7 "2017-01-04T07:02:16Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.