# Increasing shard relocation speed **URL:** \ **Category:** Elasticsearch\ **Created:** [June 27, 2016, 8:15am UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009 "2016-06-27T08:15:54Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![haizaar](https://avatars.discourse-cdn.com/v4/letter/h/3d9bf3/32.png) [@haizaar](https://discuss.elastic.co/u/haizaar)\ **Post date:** [June 27, 2016, 8:15am UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/1 "2016-06-27T08:15:54Z")
I have ES 2.3.3 running with 6 data nodes. Each index has one replica. ONE of the data nodes was down for a day. Now it's back to life and ES started relocating shards to it. The thing is that it only relocates two shards at a time. Each nodes holds about 1TB of data and it looks like it will take many hours. How can I increase this number to speed the process up? P.S. I've also set `indices.recovery.max_bytes_per_sec` to `200mb`, though I see that java process on the recovering node writes only 70-80MB/s (and I've tested my disks to provide 200+ Mb/s). Thanks! ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [June 27, 2016, 8:37am UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/2 "2016-06-27T08:37:32Z")
You could try one of those settings: [https://www.elastic.co/guide/en/elasticsearch/reference/2.3/recovery.html#recovery](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/recovery.html#recovery) - `indices.recovery.concurrent_streams` - `indices.recovery.concurrent_small_file_streams` ---
**Author:** ![haizaar](https://avatars.discourse-cdn.com/v4/letter/h/3d9bf3/32.png) [@haizaar](https://discuss.elastic.co/u/haizaar)\ **Post date:** [June 27, 2016, 8:43am UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/3 "2016-06-27T08:43:10Z")
`concurrent_streams` was already set to 5. Set `concurrent_small_file_streams` to 5 as well, but still, only two shards a time ☹ ---
**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\ **Post date:** [June 27, 2016, 12:24pm UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/4 "2016-06-27T12:24:55Z")
Have a look at the allocation decider `cluster.routing.allocation.node_concurrent_recoveries`: How many concurrent shard recoveries are allowed to happen on a node. Defaults to 2. [https://www.elastic.co/guide/en/elasticsearch/reference/2.3/shards-allocation.html](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/shards-allocation.html) ---
**Author:** ![haizaar](https://avatars.discourse-cdn.com/v4/letter/h/3d9bf3/32.png) [@haizaar](https://discuss.elastic.co/u/haizaar)\ **Post date:** [June 27, 2016, 12:39pm UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/5 "2016-06-27T12:39:52Z")
Changed to 10. My cluster settings now look like the below. To test this, I evicted one node by excluding it through `cluster.routing.allocation.exclude._ip`. However still there were only 2 relocating shards in ES at a time. I think this is because the settings you've mentioned relate to recovery, while what I'm experiencing is shard relocation. I.e. I'm joining new node to the cluster (to scale out) and only two shards at a time are being moved to it. So my original question still stands - how to boost shard relocation speed? My cluster settings: ```auto { "persistent" : { "cluster" : { "routing" : { "allocation" : { "node_concurrent_recoveries" : "10", "node_initial_primaries_recoveries" : "10" } } }, "discovery" : { "zen" : { "minimum_master_nodes" : "2" } }, "indices" : { "recovery" : { "concurrent_small_file_streams" : "5", "concurrent_streams" : "5", "max_bytes_per_sec" : "200mb" } } }, "transient" : { "cluster" : { "routing" : { "allocation" : { "enable" : "all", "exclude" : { "_ip" : "" } } } } } } ``` ---
**Author:** ![ywelsch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ywelsch/32/7751_2.png) [@ywelsch](https://discuss.elastic.co/u/ywelsch)\ **Post date:** [June 27, 2016, 12:53pm UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/6 "2016-06-27T12:53:41Z")
On the page I linked, you can find the setting `cluster.routing.allocation.cluster_concurrent_rebalance` Allow to control how many concurrent shard rebalances are allowed cluster wide. Defaults to 2. Relocations are considered as recoveries as well, so you should still increase `cluster.routing.allocation.node_concurrent_recoveries` if all relocations go to one node. ---
**Author:** ![haizaar](https://avatars.discourse-cdn.com/v4/letter/h/3d9bf3/32.png) [@haizaar](https://discuss.elastic.co/u/haizaar)\ **Post date:** [June 27, 2016, 12:56pm UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/7 "2016-06-27T12:56:31Z")
Yeehaa! Set this to 10 and it did the trick! 10 relocating shards. Thank you very much for your help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:40pm UTC](https://discuss.elastic.co/t/increasing-shard-relocation-speed/54009/8 "2017-07-05T22:40:15Z")