# Shard Balancing **URL:** \ **Category:** Elasticsearch\ **Created:** [January 20, 2012, 8:42pm UTC](https://discuss.elastic.co/t/shard-balancing/6456 "2012-01-20T20:42:22Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![jjasinek](https://avatars.discourse-cdn.com/v4/letter/j/b782af/32.png) [@jjasinek](https://discuss.elastic.co/u/jjasinek)\ **Post date:** [January 20, 2012, 8:42pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/1 "2012-01-20T20:42:22Z")
While reviewing our ElasticSearch cluster today I noticed that the shards for one of the indexes didn't appear to be evenly balanced across the nodes. After speaking with another developer, we noticed that the total number of shards, regardless of index, per node was roughly the same. This was surprising to me, as I would have assumed it would have balanced an even number of shards per index per node instead of an even number of shards per node. My concern with it doing the later is that there isn't a guarantee that the indexes themselves are of the same size or take the same amount of queries. As such you could get essentially overload one box. Take our cluster for example. At the time I noticed this it consisted of 5 indexes (5 shards and 1 replica each) and a total of 5 nodes. One of the indexes is roughly 40GB while three were about 10GB and the final index about 10MB. What I had noticed was that for the large index (and the one with the most load), 5 of the shards were on one node instead of the 2 shards per node that I would have assumed. On the smaller index I had even noticed that there were 3 shards instead of 2. I then started counting the total number of shards, regardless of index, per node, and realized that each node had 10 shards. I wouldn't normally think much of it if all of the indexes were the same size but ours aren't. I'm concerned that 50% of one index is on one node instead of being distributed evenly to five. I'm not worried about disk space now but what I'm concerned about is the distribution of searching. This one box would also take the majority of the search traffic. Another note: when I deleted some of the indexes that we no longer needed. The other indexes started to re-balance. Another indicator that balancing of shards per node is across all indexes and not per index. ---
**Author:** ![Berkay\_Mollamustafao](https://avatars.discourse-cdn.com/v4/letter/b/22d042/32.png) [@Berkay\_Mollamustafao](https://discuss.elastic.co/u/Berkay_Mollamustafao)\ **Post date:** [January 20, 2012, 9:13pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/2 "2012-01-20T21:13:24Z")
Currently in ES, distribution of shards among the nodes is indeed only based on number of shards as you have noticed. Distribution algorithm does not take the size of the shards into account. If you have indices with significantly different size, you can have the unbalanced nodes problem you're describing. Regards, Berkay Mollamustafaoglu mberkay on yahoo, google and skype On Fri, Jan 20, 2012 at 3:42 PM, jjasinek [jjasinek@gmail.com](mailto:jjasinek@gmail.com) wrote: > While reviewing our Elasticsearch cluster today I noticed that the > shards for one of the indexes didn't appear to be evenly balanced > across the nodes. After speaking with another developer, we noticed > that the total number of shards, regardless of index, per node was > roughly the same. This was surprising to me, as I would have assumed > it would have balanced an even number of shards per index per node > instead of an even number of shards per node. My concern with it > doing the later is that there isn't a guarantee that the indexes > themselves are of the same size or take the same amount of queries. > As such you could get essentially overload one box. > > Take our cluster for example. At the time I noticed this it consisted > of 5 indexes (5 shards and 1 replica each) and a total of 5 nodes. > One of the indexes is roughly 40GB while three were about 10GB and the > final index about 10MB. > > What I had noticed was that for the large index (and the one with the > most load), 5 of the shards were on one node instead of the 2 shards > per node that I would have assumed. On the smaller index I had even > noticed that there were 3 shards instead of 2. I then started > counting the total number of shards, regardless of index, per node, > and realized that each node had 10 shards. > > I wouldn't normally think much of it if all of the indexes were the > same size but ours aren't. I'm concerned that 50% of one index is on > one node instead of being distributed evenly to five. I'm not worried > about disk space now but what I'm concerned about is the distribution > of searching. This one box would also take the majority of the search > traffic. > > Another note: when I deleted some of the indexes that we no longer > needed. The other indexes started to re-balance. Another indicator > that balancing of shards per node is across all indexes and not per > index. ---
**Author:** ![Mark\_Waddle](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_waddle/32/2608_2.png) [@Mark\_Waddle](https://discuss.elastic.co/u/Mark_Waddle)\ **Post date:** [January 21, 2012, 3:55am UTC](https://discuss.elastic.co/t/shard-balancing/6456/3 "2012-01-21T03:55:37Z")
I have only been working with ES for a few weeks now, so take my advice with a grain of salt. A possible mitigation might be to have each node in the cluster bound to a specific index using the index shard allocation described here [http://www.elasticsearch.org/guide/reference/index-modules/allocation.html](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html). You could scale up or down for each index by reducing/increasing the resources per node, or increasing the number of nodes. The nodes could run individually on hosts or in sets on hosts, whichever makes sense based on the resources and redundancy needed. Mark ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 23, 2012, 6:52pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/4 "2012-01-23T18:52:45Z")
Heya, just to add: Yes, currently, elasticsearch will make sure an even number of shards are allocated across nodes, regardless of which index they belong to. On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [mark@markwaddle.com](mailto:mark@markwaddle.com) wrote: > I have only been working with ES for a few weeks now, so take my advice > with a grain of salt. > > A possible mitigation might be to have each node in the cluster bound to a > specific index using the index shard allocation described here > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation.html). > You could scale up or down for each index by reducing/increasing the > resources per node, or increasing the number of nodes. The nodes could run > individually on hosts or in sets on hosts, whichever makes sense based on > the resources and redundancy needed. > > Mark ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [January 23, 2012, 8:23pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/5 "2012-01-23T20:23:30Z")
Shay - out of curiosity - do you plan on making allocation algo pluggable or adding alternative allocation options? Thanks, Otis On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > Heya, just to add: Yes, currently, elasticsearch will make sure an even > number of shards are allocated across nodes, regardless of which index they > belong to. > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) wrote: > > > I have only been working with ES for a few weeks now, so take my advice > > with a grain of salt. > > > A possible mitigation might be to have each node in the cluster bound to a > > specific index using the index shard allocation described here > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation).... > > You could scale up or down for each index by reducing/increasing the > > resources per node, or increasing the number of nodes. The nodes could run > > individually on hosts or in sets on hosts, whichever makes sense based on > > the resources and redundancy needed. > > > Mark ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 23, 2012, 8:48pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/6 "2012-01-23T20:48:09Z")
You can control where indices are placed and how replicas are distributed externally already, see more here: [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/cluster.html) (shard allocation awareness and filtering). There is a class that deals just with how to balance shards, its called EvenShardsCountAllocator ( [https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/EvenShardsCountAllocator.java](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/EvenShardsCountAllocator.java)) and implements ShardsAllocator. It can be easily allowed to be pluggable if needed. Note, the tricky bit here is to have a balancing logic that moves as little shards as possible while still providing the best distribution. On Mon, Jan 23, 2012 at 10:23 PM, Otis Gospodnetic \< [otis.gospodnetic@gmail.com](mailto:otis.gospodnetic@gmail.com)\> wrote: > Shay - out of curiosity - do you plan on making allocation algo > pluggable or adding alternative allocation options? > > Thanks, > Otis > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > Heya, just to add: Yes, currently, elasticsearch will make sure an even > > number of shards are allocated across nodes, regardless of which index > > they > > belong to. > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > wrote: > > > > > I have only been working with ES for a few weeks now, so take my advice > > > with a grain of salt. > > > > > A possible mitigation might be to have each node in the cluster bound > > > to a > > > specific index using the index shard allocation described here > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation).. > > > .. > > > You could scale up or down for each index by reducing/increasing the > > > resources per node, or increasing the number of nodes. The nodes could > > > run > > > individually on hosts or in sets on hosts, whichever makes sense based > > > on > > > the resources and redundancy needed. > > > > > Mark ---
**Author:** ![Yooz](https://avatars.discourse-cdn.com/v4/letter/y/13edae/32.png) [@Yooz](https://discuss.elastic.co/u/Yooz)\ **Post date:** [January 23, 2012, 8:48pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/7 "2012-01-23T20:48:43Z")
You can always create your own allocation module under: modules/elasticsearch/src/main/java/org/elasticsearch/cluster/routing/ allocation/allocator/ We are running a custom module that balances approximately within an index. I think in general though, there is a plan to make more resource aware allocation schemes, i.e. not only index size, but also cpu/memory/disk constraints across non-uniform hardware, which is a superset of this issue. On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) wrote: > Shay - out of curiosity - do you plan on making allocation algo > pluggable or adding alternative allocation options? > > Thanks, > Otis > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > Heya, just to add: Yes, currently, elasticsearch will make sure an even > > number of shards are allocated across nodes, regardless of which index they > > belong to. > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) wrote: > > > > > I have only been working with ES for a few weeks now, so take my advice > > > with a grain of salt. > > > > A possible mitigation might be to have each node in the cluster bound to a > > > specific index using the index shard allocation described here > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation).... > > > You could scale up or down for each index by reducing/increasing the > > > resources per node, or increasing the number of nodes. The nodes could run > > > individually on hosts or in sets on hosts, whichever makes sense based on > > > the resources and redundancy needed. > > > > Mark ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 23, 2012, 8:51pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/8 "2012-01-23T20:51:02Z")
@Yooz: cool!, can you share the code? People might find it helpful. On Mon, Jan 23, 2012 at 10:48 PM, Yooz [youngmaeng@gmail.com](mailto:youngmaeng@gmail.com) wrote: > You can always create your own allocation module under: > modules/elasticsearch/src/main/java/org/elasticsearch/cluster/routing/ > allocation/allocator/ > > We are running a custom module that balances approximately within an > index. I think in general though, there is a plan to make more > resource aware allocation schemes, i.e. not only index size, but also > cpu/memory/disk constraints across non-uniform hardware, which is a > superset of this issue. > > On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) > wrote: > > > Shay - out of curiosity - do you plan on making allocation algo > > pluggable or adding alternative allocation options? > > > > Thanks, > > Otis > > > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > Heya, just to add: Yes, currently, elasticsearch will make sure an even > > > number of shards are allocated across nodes, regardless of which index > > > they > > > belong to. > > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > > wrote: > > > > > > > I have only been working with ES for a few weeks now, so take my > > > > advice > > > > with a grain of salt. > > > > > > A possible mitigation might be to have each node in the cluster > > > > bound to a > > > > specific index using the index shard allocation described here > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation).... > > > > > You could scale up or down for each index by reducing/increasing the > > > > resources per node, or increasing the number of nodes. The nodes > > > > could run > > > > individually on hosts or in sets on hosts, whichever makes sense > > > > based on > > > > the resources and redundancy needed. > > > > > > Mark ---
**Author:** ![MagmaRules](https://avatars.discourse-cdn.com/v4/letter/m/bcef8e/32.png) [@MagmaRules](https://discuss.elastic.co/u/MagmaRules)\ **Post date:** [May 21, 2012, 5:28pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/9 "2012-05-21T17:28:27Z")
Hi there, @Yooz: Can you share some basic steps on how you implemented your allocator? I'm currently facing this problem. I have an empty index that ES is giving the same relevance as another index that has 70GB of size. On Monday, January 23, 2012 8:51:02 PM UTC, kimchy wrote: > @Yooz: cool!, can you share the code? People might find it helpful. > > On Mon, Jan 23, 2012 at 10:48 PM, Yooz [youngmaeng@gmail.com](mailto:youngmaeng@gmail.com) wrote: > > > You can always create your own allocation module under: > > modules/elasticsearch/src/main/java/org/elasticsearch/cluster/routing/ > > allocation/allocator/ > > > > We are running a custom module that balances approximately within an > > index. I think in general though, there is a plan to make more > > resource aware allocation schemes, i.e. not only index size, but also > > cpu/memory/disk constraints across non-uniform hardware, which is a > > superset of this issue. > > > > On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) > > wrote: > > > > > Shay - out of curiosity - do you plan on making allocation algo > > > pluggable or adding alternative allocation options? > > > > > > Thanks, > > > Otis > > > > > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > Heya, just to add: Yes, currently, elasticsearch will make sure an > > > > even > > > > number of shards are allocated across nodes, regardless of which > > > > index they > > > > belong to. > > > > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > > > wrote: > > > > > > > > > I have only been working with ES for a few weeks now, so take my > > > > > advice > > > > > with a grain of salt. > > > > > > > > A possible mitigation might be to have each node in the cluster > > > > > bound to a > > > > > specific index using the index shard allocation described here > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation).... > > > > > > > You could scale up or down for each index by reducing/increasing the > > > > > resources per node, or increasing the number of nodes. The nodes > > > > > could run > > > > > individually on hosts or in sets on hosts, whichever makes sense > > > > > based on > > > > > the resources and redundancy needed. > > > > > > > > Mark On Monday, January 23, 2012 8:51:02 PM UTC, kimchy wrote: > @Yooz: cool!, can you share the code? People might find it helpful. > > On Mon, Jan 23, 2012 at 10:48 PM, Yooz [youngmaeng@gmail.com](mailto:youngmaeng@gmail.com) wrote: > > > You can always create your own allocation module under: > > modules/elasticsearch/src/main/java/org/elasticsearch/cluster/routing/ > > allocation/allocator/ > > > > We are running a custom module that balances approximately within an > > index. I think in general though, there is a plan to make more > > resource aware allocation schemes, i.e. not only index size, but also > > cpu/memory/disk constraints across non-uniform hardware, which is a > > superset of this issue. > > > > On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) > > wrote: > > > > > Shay - out of curiosity - do you plan on making allocation algo > > > pluggable or adding alternative allocation options? > > > > > > Thanks, > > > Otis > > > > > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > Heya, just to add: Yes, currently, elasticsearch will make sure an > > > > even > > > > number of shards are allocated across nodes, regardless of which > > > > index they > > > > belong to. > > > > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > > > wrote: > > > > > > > > > I have only been working with ES for a few weeks now, so take my > > > > > advice > > > > > with a grain of salt. > > > > > > > > A possible mitigation might be to have each node in the cluster > > > > > bound to a > > > > > specific index using the index shard allocation described here > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/allocation).... > > > > > > > You could scale up or down for each index by reducing/increasing the > > > > > resources per node, or increasing the number of nodes. The nodes > > > > > could run > > > > > individually on hosts or in sets on hosts, whichever makes sense > > > > > based on > > > > > the resources and redundancy needed. > > > > > > > > Mark ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [May 21, 2012, 6:00pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/10 "2012-05-21T18:00:37Z")
+1 ---
**Author:** ![John\_Cwikla\_2](https://avatars.discourse-cdn.com/v4/letter/j/54ee81/32.png) [@John\_Cwikla\_2](https://discuss.elastic.co/u/John_Cwikla_2)\ **Post date:** [May 21, 2012, 9:32pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/11 "2012-05-21T21:32:54Z")
I had the same problem where I had a tiny index causing the shards of a large index to be balanced oddly. I ended up setting the small indexe's replicas = shards -1, which will force the larger index to balance as if there were no other indexes 🙂 On Mon, May 21, 2012 at 10:28 AM, MagmaRules [mfcoxo@gmail.com](mailto:mfcoxo@gmail.com) wrote: > Hi there, > > @Yooz: Can you share some basic steps on how you implemented your > allocator? > > I'm currently facing this problem. I have an empty index that ES is giving > the same relevance as another index that has 70GB of size. > > On Monday, January 23, 2012 8:51:02 PM UTC, kimchy wrote: > > > @Yooz: cool!, can you share the code? People might find it helpful. > > > > On Mon, Jan 23, 2012 at 10:48 PM, Yooz [youngmaeng@gmail.com](mailto:youngmaeng@gmail.com) wrote: > > > > > You can always create your own allocation module under: > > > modules/elasticsearch/src/ **main/java/org/elasticsearch/** > > > cluster/routing/ > > > allocation/allocator/ > > > > > > We are running a custom module that balances approximately within an > > > index. I think in general though, there is a plan to make more > > > resource aware allocation schemes, i.e. not only index size, but also > > > cpu/memory/disk constraints across non-uniform hardware, which is a > > > superset of this issue. > > > > > > On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) > > > wrote: > > > > > > > Shay - out of curiosity - do you plan on making allocation algo > > > > pluggable or adding alternative allocation options? > > > > > > > > Thanks, > > > > Otis > > > > > > > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > > > Heya, just to add: Yes, currently, elasticsearch will make sure an > > > > > even > > > > > number of shards are allocated across nodes, regardless of which > > > > > index they > > > > > belong to. > > > > > > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > > > > wrote: > > > > > > > > > > > I have only been working with ES for a few weeks now, so take my > > > > > > advice > > > > > > with a grain of salt. > > > > > > > > > > A possible mitigation might be to have each node in the cluster > > > > > > bound to a > > > > > > specific index using the index shard allocation described here > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > > > allocation..[http://www.elasticsearch.org/guide/reference/index-modules/allocation..](http://www.elasticsearch.org/guide/reference/index-modules/allocation..) > > > > > > .. > > > > > > You could scale up or down for each index by reducing/increasing > > > > > > the > > > > > > resources per node, or increasing the number of nodes. The nodes > > > > > > could run > > > > > > individually on hosts or in sets on hosts, whichever makes sense > > > > > > based on > > > > > > the resources and redundancy needed. > > > > > > > > > > Mark > > On Monday, January 23, 2012 8:51:02 PM UTC, kimchy wrote: > > > @Yooz: cool!, can you share the code? People might find it helpful. > > > > On Mon, Jan 23, 2012 at 10:48 PM, Yooz [youngmaeng@gmail.com](mailto:youngmaeng@gmail.com) wrote: > > > > > You can always create your own allocation module under: > > > modules/elasticsearch/src/ **main/java/org/elasticsearch/** > > > cluster/routing/ > > > allocation/allocator/ > > > > > > We are running a custom module that balances approximately within an > > > index. I think in general though, there is a plan to make more > > > resource aware allocation schemes, i.e. not only index size, but also > > > cpu/memory/disk constraints across non-uniform hardware, which is a > > > superset of this issue. > > > > > > On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) > > > wrote: > > > > > > > Shay - out of curiosity - do you plan on making allocation algo > > > > pluggable or adding alternative allocation options? > > > > > > > > Thanks, > > > > Otis > > > > > > > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > > > Heya, just to add: Yes, currently, elasticsearch will make sure an > > > > > even > > > > > number of shards are allocated across nodes, regardless of which > > > > > index they > > > > > belong to. > > > > > > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > > > > wrote: > > > > > > > > > > > I have only been working with ES for a few weeks now, so take my > > > > > > advice > > > > > > with a grain of salt. > > > > > > > > > > A possible mitigation might be to have each node in the cluster > > > > > > bound to a > > > > > > specific index using the index shard allocation described here > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > > > allocation..[http://www.elasticsearch.org/guide/reference/index-modules/allocation..](http://www.elasticsearch.org/guide/reference/index-modules/allocation..) > > > > > > .. > > > > > > You could scale up or down for each index by reducing/increasing > > > > > > the > > > > > > resources per node, or increasing the number of nodes. The nodes > > > > > > could run > > > > > > individually on hosts or in sets on hosts, whichever makes sense > > > > > > based on > > > > > > the resources and redundancy needed. > > > > > > > > > > Mark ---
**Author:** ![John\_Cwikla\_2](https://avatars.discourse-cdn.com/v4/letter/j/54ee81/32.png) [@John\_Cwikla\_2](https://discuss.elastic.co/u/John_Cwikla_2)\ **Post date:** [May 21, 2012, 9:39pm UTC](https://discuss.elastic.co/t/shard-balancing/6456/12 "2012-05-21T21:39:32Z")
Woops, that should have been "nodes" not "shards" - 1. On Mon, May 21, 2012 at 2:32 PM, John Cwikla [cwikla@radiusintel.com](mailto:cwikla@radiusintel.com) wrote: > I had the same problem where I had a tiny index causing the shards of a > large index to be balanced oddly. > I ended up setting the small indexe's replicas = shards -1, which will > force the larger index to balance > as if there were no other indexes 🙂 > > On Mon, May 21, 2012 at 10:28 AM, MagmaRules [mfcoxo@gmail.com](mailto:mfcoxo@gmail.com) wrote: > > > Hi there, > > > > @Yooz: Can you share some basic steps on how you implemented your > > allocator? > > > > I'm currently facing this problem. I have an empty index that ES is > > giving the same relevance as another index that has 70GB of size. > > > > On Monday, January 23, 2012 8:51:02 PM UTC, kimchy wrote: > > > > > @Yooz: cool!, can you share the code? People might find it helpful. > > > > > > On Mon, Jan 23, 2012 at 10:48 PM, Yooz [youngmaeng@gmail.com](mailto:youngmaeng@gmail.com) wrote: > > > > > > > You can always create your own allocation module under: > > > > modules/elasticsearch/src/ **main/java/org/elasticsearch/** > > > > cluster/routing/ > > > > allocation/allocator/ > > > > > > > > We are running a custom module that balances approximately within an > > > > index. I think in general though, there is a plan to make more > > > > resource aware allocation schemes, i.e. not only index size, but also > > > > cpu/memory/disk constraints across non-uniform hardware, which is a > > > > superset of this issue. > > > > > > > > On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) > > > > wrote: > > > > > > > > > Shay - out of curiosity - do you plan on making allocation algo > > > > > pluggable or adding alternative allocation options? > > > > > > > > > > Thanks, > > > > > Otis > > > > > > > > > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > > > > > Heya, just to add: Yes, currently, elasticsearch will make sure an > > > > > > even > > > > > > number of shards are allocated across nodes, regardless of which > > > > > > index they > > > > > > belong to. > > > > > > > > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > > > > > wrote: > > > > > > > > > > > > > I have only been working with ES for a few weeks now, so take my > > > > > > > advice > > > > > > > with a grain of salt. > > > > > > > > > > > > A possible mitigation might be to have each node in the cluster > > > > > > > bound to a > > > > > > > specific index using the index shard allocation described here > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > > > > allocation..[http://www.elasticsearch.org/guide/reference/index-modules/allocation..](http://www.elasticsearch.org/guide/reference/index-modules/allocation..) > > > > > > > .. > > > > > > > You could scale up or down for each index by reducing/increasing > > > > > > > the > > > > > > > resources per node, or increasing the number of nodes. The nodes > > > > > > > could run > > > > > > > individually on hosts or in sets on hosts, whichever makes sense > > > > > > > based on > > > > > > > the resources and redundancy needed. > > > > > > > > > > > > Mark > > > > On Monday, January 23, 2012 8:51:02 PM UTC, kimchy wrote: > > > > > @Yooz: cool!, can you share the code? People might find it helpful. > > > > > > On Mon, Jan 23, 2012 at 10:48 PM, Yooz [youngmaeng@gmail.com](mailto:youngmaeng@gmail.com) wrote: > > > > > > > You can always create your own allocation module under: > > > > modules/elasticsearch/src/ **main/java/org/elasticsearch/** > > > > cluster/routing/ > > > > allocation/allocator/ > > > > > > > > We are running a custom module that balances approximately within an > > > > index. I think in general though, there is a plan to make more > > > > resource aware allocation schemes, i.e. not only index size, but also > > > > cpu/memory/disk constraints across non-uniform hardware, which is a > > > > superset of this issue. > > > > > > > > On Jan 23, 12:23 pm, Otis Gospodnetic [otis.gospodne...@gmail.com](mailto:otis.gospodne...@gmail.com) > > > > wrote: > > > > > > > > > Shay - out of curiosity - do you plan on making allocation algo > > > > > pluggable or adding alternative allocation options? > > > > > > > > > > Thanks, > > > > > Otis > > > > > > > > > > On Jan 23, 1:52 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > > > > > > Heya, just to add: Yes, currently, elasticsearch will make sure an > > > > > > even > > > > > > number of shards are allocated across nodes, regardless of which > > > > > > index they > > > > > > belong to. > > > > > > > > > > > On Sat, Jan 21, 2012 at 5:55 AM, Mark Waddle [m...@markwaddle.com](mailto:m...@markwaddle.com) > > > > > > wrote: > > > > > > > > > > > > > I have only been working with ES for a few weeks now, so take my > > > > > > > advice > > > > > > > with a grain of salt. > > > > > > > > > > > > A possible mitigation might be to have each node in the cluster > > > > > > > bound to a > > > > > > > specific index using the index shard allocation described here > > > > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/**guide/reference/index-modules/)\*\* > > > > > > > allocation..[http://www.elasticsearch.org/guide/reference/index-modules/allocation..](http://www.elasticsearch.org/guide/reference/index-modules/allocation..) > > > > > > > .. > > > > > > > You could scale up or down for each index by reducing/increasing > > > > > > > the > > > > > > > resources per node, or increasing the number of nodes. The nodes > > > > > > > could run > > > > > > > individually on hosts or in sets on hosts, whichever makes sense > > > > > > > based on > > > > > > > the resources and redundancy needed. > > > > > > > > > > > > Mark ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:27am UTC](https://discuss.elastic.co/t/shard-balancing/6456/13 "2017-07-06T03:27:46Z")