Currently in ES, distribution of shards among the nodes is indeed only
based on number of shards as you have noticed. Distribution algorithm does
not take the size of the shards into account. If you have indices with
significantly different size, you can have the unbalanced nodes problem
you're describing.
Regards,
Berkay Mollamustafaoglu
mberkay on yahoo, google and skype
On Fri, Jan 20, 2012 at 3:42 PM, jjasinek jjasinek@gmail.com wrote:
While reviewing our Elasticsearch cluster today I noticed that the
shards for one of the indexes didn't appear to be evenly balanced
across the nodes. After speaking with another developer, we noticed
that the total number of shards, regardless of index, per node was
roughly the same. This was surprising to me, as I would have assumed
it would have balanced an even number of shards per index per node
instead of an even number of shards per node. My concern with it
doing the later is that there isn't a guarantee that the indexes
themselves are of the same size or take the same amount of queries.
As such you could get essentially overload one box.Take our cluster for example. At the time I noticed this it consisted
of 5 indexes (5 shards and 1 replica each) and a total of 5 nodes.
One of the indexes is roughly 40GB while three were about 10GB and the
final index about 10MB.What I had noticed was that for the large index (and the one with the
most load), 5 of the shards were on one node instead of the 2 shards
per node that I would have assumed. On the smaller index I had even
noticed that there were 3 shards instead of 2. I then started
counting the total number of shards, regardless of index, per node,
and realized that each node had 10 shards.I wouldn't normally think much of it if all of the indexes were the
same size but ours aren't. I'm concerned that 50% of one index is on
one node instead of being distributed evenly to five. I'm not worried
about disk space now but what I'm concerned about is the distribution
of searching. This one box would also take the majority of the search
traffic.Another note: when I deleted some of the indexes that we no longer
needed. The other indexes started to re-balance. Another indicator
that balancing of shards per node is across all indexes and not per
index.