Thanks for ElasticSearch, it's grand.
I'm running ES 0.18.6, under Sun's JDK in Ubuntu 11.04 on EC2, in a 6-
node cluster, near-default config: only setting cluster.name,
network.host, discovery.zen.ping.unicast.hosts, and path.data. None of
the shard allocation or rebalancing settings have been touched. The
cluster is set up to use a single index, so 10 shards are in play.
Whenever this index is (re)created, it appears as evenly allocated as
it can be for that configuration, with 1 or 2 shards per node. That's
what I'd expect to always be the case when rebalancing is complete.
However, when rebalancing does occur (nodes entering/leaving the
cluster) I frequently end up unbalanced, with most nodes only holding
1 shard and one or two of them holding 3 or 4 shards.
Is this a known behavior of the current algorithm? Everything I've
read in the docs + mailing list says that balancing is supposed to go
by number of shards per node (vs e.g the oft-discussed balancing by
data weight), but I wonder if I am misunderstanding something.