I'm wondering how are you guys doing with data rebalancing on large clusters ? We have 3 masters, 3 hot nodes and 6 cold. Some cold nodes have 4T disk and few 2T. ES is rebalancing shards based on number shards per node so time to time we need manually relocated heavy shards between nodes. We are not able control shards size, some are 200G some 10G. We are still on ES 6.8 and we are planning upgrade to 7.x where we can use index lifecycle management.
I read about plugin that enables rebalancing based on disk usage, another option is cron job and script to rebalance data.
What is your approach and why you choose it ?