# Rebalancing a cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [August 22, 2012, 9:21pm UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821 "2012-08-22T21:21:03Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![James\_Bardin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_bardin/32/2536_2.png) [@James\_Bardin](https://discuss.elastic.co/u/James_Bardin)\ **Post date:** [August 22, 2012, 9:21pm UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821/1 "2012-08-22T21:21:03Z")
I have a cluster with one index which is much more heavily used than the rest. My problem is keeping the busy index distributed across all nodes. For example, do to recent changes in the the cluster, I now have 1 node with 80 percent of the hot primary shards, and two nodes doing almost nothing. I been using "i.r.a.include.name" to force elasticsearch to move shards around, but since it always wants to evenly distribute the number of shards, the load often ends up skewed. This can work, but it take a few pushes to get the shards to fall nicely, and isn't a maintainable solution (also if anyone knows how to _unset_ an index setting, I'd like to get rid of the "index.routing.allocation" settings when I'm done, so they don't need to be updated if the cluster gets resized). So, is there a better way to balance a cluster? Should I just double or tripple the shard count, so they're more likely to get evenly distributed? Thanks, -jim -- ---
**Author:** ![Mohammady\_Mahdy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammady_mahdy/32/2128_2.png) [@Mohammady\_Mahdy](https://discuss.elastic.co/u/Mohammady_Mahdy)\ **Post date:** [January 16, 2013, 9:14am UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821/2 "2013-01-16T09:14:27Z")
Did you find any solution for this? On Thursday, August 23, 2012 1:21:03 AM UTC+4, James Bardin wrote: > I have a cluster with one index which is much more heavily used than the > rest. My problem is keeping the busy index distributed across all nodes. > For example, do to recent changes in the the cluster, I now have 1 node > with 80 percent of the hot primary shards, and two nodes doing almost > nothing. > > I been using "i.r.a.include.name" to force elasticsearch to move shards > around, but since it always wants to evenly distribute the number of > shards, the load often ends up skewed. This can work, but it take a few > pushes to get the shards to fall nicely, and isn't a maintainable solution > (also if anyone knows how to _unset_ an index setting, I'd like to get rid > of the "index.routing.allocation" settings when I'm done, so they don't > need to be updated if the cluster gets resized). > > So, is there a better way to balance a cluster? > Should I just double or tripple the shard count, so they're more likely to > get evenly distributed? > > Thanks, > -jim -- ---
**Author:** ![ferhatsb](https://avatars.discourse-cdn.com/v4/letter/f/977dab/32.png) [@ferhatsb](https://discuss.elastic.co/u/ferhatsb)\ **Post date:** [January 16, 2013, 1:56pm UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821/3 "2013-01-16T13:56:33Z")
Hi, Maybe you can set "index.routing.allocation.total\_shards\_per\_node" for that nasty index. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Regards, Ferhat On Thursday, August 23, 2012 12:21:03 AM UTC+3, James Bardin wrote: > I have a cluster with one index which is much more heavily used than the > rest. My problem is keeping the busy index distributed across all nodes. > For example, do to recent changes in the the cluster, I now have 1 node > with 80 percent of the hot primary shards, and two nodes doing almost > nothing. > > I been using "i.r.a.include.name" to force elasticsearch to move shards > around, but since it always wants to evenly distribute the number of > shards, the load often ends up skewed. This can work, but it take a few > pushes to get the shards to fall nicely, and isn't a maintainable solution > (also if anyone knows how to _unset_ an index setting, I'd like to get rid > of the "index.routing.allocation" settings when I'm done, so they don't > need to be updated if the cluster gets resized). > > So, is there a better way to balance a cluster? > Should I just double or tripple the shard count, so they're more likely to > get evenly distributed? > > Thanks, > -jim -- ---
**Author:** ![Mohammady\_Mahdy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammady_mahdy/32/2128_2.png) [@Mohammady\_Mahdy](https://discuss.elastic.co/u/Mohammady_Mahdy)\ **Post date:** [January 16, 2013, 2:30pm UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821/4 "2013-01-16T14:30:29Z")
Oh, very nice. On Wed, Jan 16, 2013 at 5:56 PM, ferhatsb [ferhat.sobay@gmail.com](mailto:ferhat.sobay@gmail.com) wrote: > Hi, > > Maybe you can set "index.routing.allocation.total\_shards\_per\_node" for > that nasty index. > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > Regards, > Ferhat > > On Thursday, August 23, 2012 12:21:03 AM UTC+3, James Bardin wrote: > > > I have a cluster with one index which is much more heavily used than the > > rest. My problem is keeping the busy index distributed across all nodes. > > For example, do to recent changes in the the cluster, I now have 1 node > > with 80 percent of the hot primary shards, and two nodes doing almost > > nothing. > > > > I been using "i.r.a.include.name" to force elasticsearch to move shards > > around, but since it always wants to evenly distribute the number of > > shards, the load often ends up skewed. This can work, but it take a few > > pushes to get the shards to fall nicely, and isn't a maintainable solution > > (also if anyone knows how to _unset_ an index setting, I'd like to get rid > > of the "index.routing.allocation" settings when I'm done, so they don't > > need to be updated if the cluster gets resized). > > > > So, is there a better way to balance a cluster? > > Should I just double or tripple the shard count, so they're more likely > > to get evenly distributed? > > > > Thanks, > > -jim > > -- -- ---
**Author:** ![James\_Bardin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/james_bardin/32/2536_2.png) [@James\_Bardin](https://discuss.elastic.co/u/James_Bardin)\ **Post date:** [January 16, 2013, 4:19pm UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821/5 "2013-01-16T16:19:47Z")
On Wed, Jan 16, 2013 at 8:56 AM, ferhatsb [ferhat.sobay@gmail.com](mailto:ferhat.sobay@gmail.com) wrote: > Maybe you can set "index.routing.allocation.total\_shards\_per\_node" for > that nasty index. > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) I wish the elasticsearch docs tagged features in the docs with their release. I don't think this was in the stable version when I first asked, but I'm not sure now. I ended up having to schedule a full cluster shutdown to clear this up. Also, last time I checked, there still is no way to remove index settings without restarting, but many of them now treat an empty string as if it were unset. -jim -- ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [January 16, 2013, 9:43pm UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821/6 "2013-01-16T21:43:10Z")
If you're not sure, search google/github will typically bring you to the issue that added the feature: > > > Allow to set the maximum number of shards allowed per node for a specific index …by setting: \`index.routing.allocation.total\_shards\_per\_node\`. This setting can be changed dynamically using the index update settings API, both to enable it (setting it to a value higher than 0), or disabel it (setting it to -1). Not ideal, but it works. On Wednesday, January 16, 2013 9:19:47 AM UTC-7, James Bardin wrote: > On Wed, Jan 16, 2013 at 8:56 AM, ferhatsb \<[ferhat...@gmail.com](mailto:ferhat...@gmail.com)\ > > > wrote: > > > Maybe you can set "index.routing.allocation.total\_shards\_per\_node" for > > that nasty index. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html) > > I wish the elasticsearch docs tagged features in the docs with their > release. I don't think this was in the stable version when I first asked, > but I'm not sure now. > > I ended up having to schedule a full cluster shutdown to clear this up. > Also, last time I checked, there still is no way to remove index settings > without restarting, but many of them now treat an empty string as if it > were unset. > > -jim -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:56am UTC](https://discuss.elastic.co/t/rebalancing-a-cluster/8821/7 "2017-07-06T02:56:03Z")