Need some clarification on cluster level shard allocation.
I have setup shard allocation using rack strategy. Have two racks (each represent 5 nodes).
Have set the attribute. (No Forced Allocation)
cluster.routing.allocation.awareness.attributes: <>
Observation - Clarification query
-
Cluster going to Yellow state with shards not being allocated, though few nodes in the rack that has primary shards has more space. The node is not same is as the one where primary actually resides rather another node in the same rack.
"As per my understanding of the document "https://www.elastic.co/guide/en/elasticsearch/reference/6.2/allocation-awareness.html", when forced allocation is not set, the allocation should fall back to the same rack when it is not able to allocate shard. Documentation says that when node cannot be contacted, but, shouldn't it also apply when it cannot allocate as the node in second rack does not have space? -
Shard balancing when some indices/shard has higher store size against some other. I cannot combine indices as-of now as they are purposed differently. Though looking at options. Regardless, the query is does the shard allocation go by number of shards (or) by size of shards. Looking at the cluster it goes by number and not size - as some nodes have higher data disk space utilized and some have less. <<This obviously cause allocation failures at times leading to non-green cluster state). Shouldn't cluster auto-rebalance by size? Trying to avoid manual forced re-allocation. "cluster.routing.rebalance.enable" is against its default value "all".
Thanks for clarification.