We have a relatively large cluster and one consistent issue we have seen from time to time is inconsistent disk usage balancing because ElasticSearch is balancing by shard count rather than shard resource consumption. Basically we will see all nodes have similar shard counts as expected however a few nodes might have been favored for small shards or 0 doc indexes. While I can address the 0 doc indexes relatively easily, the small indexes/shards are somewhat purposeful in that ILM will age that data out according to our expected retention. (So I do not just want to just try and make all shards equal in size)
Does anyone have some easy to consume resources for more efficiently balancing on disk usage as well?