What should I do in case of a complete failure during the upgrade process?

[Leonardo_Henrique]

Hello, everyone. How have you been?

I have a question related to the upgrade process. In docs (and viewing some posts here in the forum), it is said that the concept of rollback does not exist in Elasticsearch, but we have the possibility to make a snapshot restore taken in some time in the past.

But in reference it is said that "You can only restore a snapshot to a running cluster with an elected master node.".

What if:

1. for some unexpected reason all my master nodes fail during the process?
2. I am upgrading a single-master cluster and the upgrade process fails?

My first thought is making some other node a master node (or creating a new one in case of single-master cluster) but I am not sure if it is possible during a complete master nodes fail.

Is there any official tip in this sense? Or better ideas to handle these scenarios?

Thank you!!

[Christian_Dahlqvist]

@Leonardo_Henrique

for some unexpected reason all my master nodes fail during the process?

Assuming you are on a recent version of Elasticsearch I believe you will need to set up a new empty cluster and restore a snapshot using the restore API.

@Leonardo_Henrique

I am upgrading a single-master cluster and the upgrade process fails?

You should always look to have 3 master eligible nodes in any cluster. If a majority of master nodes have failed and can not be brought up you will, as in the previous scenario, need to set up a new empty cluster and restore a snapshot.