Hello,
Still in 5.6 ES version...
I am wondering, when a node is included into a cluster, how to avoid that this one gets all new index / shards, regarding the fact that he is empty or unbalanced versus other nodes : it implies very slow indexing...
For example, I have 30 nodes with 500 shards, and the new one is re-included with only 100 shards : the behaviour I saw is that this node will be filled until he is balanced with the other nodes. And, as index are renewed every day, all index about a day will be stored in only one node...
So, I was watching for a solution, where, while the cluster is still unbalanced, it does not target the skewed node.
I read that ES7 and ILM seem to answer this issue, but in ES5.6...?
Thanks 
Elasticsearch should rebalance the nodes so they have similar number of shards on each. The only time I've seen this not happen is if a new Lucerne version is introduced, then shards are only created on nodes with the latest version.
There are cluster settings that effect rebalancing, but I think something would have been changed to break it.
5.X is EOL, please upgrade with some urgency 
There has been a tonne of improvements around allocation/balancing/recovery etc in 6.X and 7.X, so upgrading will also help there.
Hi @warkolm
Yes, I know... we are working on it
I just need to understand what is going on and confirm how our old version works and the beneficits we will have with ES 7 (the version we target) = our cluster is huge and this project need to be explained and justified
Can you confirm it is "normal" with ES 5.6, that an unbalanced node will take at midnight all daily index and then indexing, reads, etc. will be performed on only one data node leading to a pretty bad day ? Also, with ES 7, do you have a link to something who would tell me how the new rebalance algorithm works ?
Thanks a lot, have a nice day.
Regards.
Sébastien
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.