Hi,
So I've got a cluster with 3 nodes running 5.6.6. I run 1 replica shard and have passed 66% disk use on all nodes.
I have no capacity to increase storage on the existing nodes so plan to renew the hardware for boxes that have double the capacity and the ability to expand in the future also. I want to do this 1 node at a time and wanted to know what is the best way to do this?
I did read about decomissioning a node however as I'm over 66% of disk used there isn't the drive capacity left in the cluster to keep a master and replica shard for all indexes in the event of a single node being removed.
As my licesne is for 3 nodes even though I will have the new node ready to slot in before I remove the old one I don't beleive that the cluster would accept the new node till I extract one.
I've not run test to see how this would work but I'm wondering if I can just build the new server on the bench, give it the same name etc, power down the old box, power up the new one, change the IP to match the one that just powered down and let the cluster "discover" that the old node has suddenly lost all it's data and let it replicate over.
Any insights or advice greatly appreceated.
Kind regards
Ant