I was looking for a setting to stop the big old shards that contain data from moving, when a new node is added.
I found on this page https://www.elastic.co/guide/en/elasticsearch/reference/current/shards-allocation.html
two setting 'cluster.routing.allocation.enable' and 'cluster.routing.rebalance.enable'
I set allocation to new_primaries and sure, the old shard stopped moving around.
I naively assumed that allocation is for moving shards and rebalance is for moving data between shards, but when I tried setting rebalance to 'none' the shards stopped moving too.
I tried reading this https://www.elastic.co/blog/every-shard-deserves-a-home
but it seems to use rebalancing as a synonym to reallocation.
So my questions are:
1)What is the difference between allocation.enable setting and rebalance.enable setting.
2)What would be a better setting to set to avoid moving old shards when an empty node comes online (but maybe not when a watermark.high is reached?)
3)Can the data in the shards of a single index become unbalanced (one shard is much bigger than another) and would ES try to fix it, or does it only move shards around?