Migration Question/Advice [HP->Dell, Centos->Debian, 7.17->8.15]

Elastic Stack Elasticsearch
1

Hi Team,

I have a three node cluster running the latest version of 7.17. I've run the migration assistant and we're clean. Bad news is its running on ancient H/W with centos7. All three machines run master and data roles.

I have three new machines, better in every way to the old ones. I've installed Debian 12.6 on them and ES 8.15 [but not started it or messed with the config in any way].

I've been asked to migrate off the old setup to new without shutting the stack down. According to my research, that's quite possible ... infact there seem to be a number of ways of achieving this.

What I'm thinking of doing is adding the three new machines to the existing cluster, allow elasticsearch to do its thing and migrate [some] shards on to the new machines, then exclude the old machines from the cluster one by one, allowing the cluster to rebuild each time.

Is it really that simple, or am I about to make a huge mistake?

My main worry is that we'll be running 7.17 and 8.15 on the same cluster, but that's what happens when you do an upgrade-in-place, so we should be fine ... right?

Any advice most gratefully accepted.

Thanks!

2

Not really, this has some risks.

You are not doing just a rolling upgrade, you are adding new nodes to replace old nodes and your nodes are both data and master nodes.

In a rolling upgrade process master nodes are the last one to be upgraded because older nodes cannot join a cluster where the master node is on a higher version.

If for some reason during this process your master changes to one of the version 8.15, and one of the older nodes disconnect from the cluster, it will not be able to join the cluster anymore unless it is upgraded to the same version.

I would not try to do both things at the same time.

You should first add your 3 new nodes on the same version of the current ones, after they are part of the same cluster, exclude one of the nodes from the shard allocation process and wait for the shards to move out to the other nodes.

After the node is empty, shut it down and exclude another one of the older nodes, repeat the process until all older nodes are out of the cluster.

You can now do a rolling upgrade your new cluster to 8.15.

While this can be done without downtime, I strongly recommend asking for a maintenance window to perform this.

3

Thanks for your advice - much appreciated.