Removing master node permanently

[artechkey]

Hi,
We currently have a 2 node + master-voting only node cluster. We are expanding the cluster by adding 3 more nodes to it. As part of the expansion, we want to designate one of the new nodes as a master and take out the master-voting-only node from the cluster.

Current config: 2 nodes (all roles) + 1 master-voting-only node
Desired config: 3 nodes (all roles) + 2 nodes (all roles except master)

From the docs, it looks like we can remove the master-voting-only node by just shutting it down. But will the cluster continue to remember this node? After removing it and adding the new nodes, there will continue to be 3 master nodes. Will these 3 form the new ensemble and any 2 of these would be sufficient for quorum in the future?

Also, is there a way to allocate all roles except "master" to a node other than explicitly specifying all the roles in the yml file?

Thanks,
AR.

[DavidTurner]

@artechkey

After removing it and adding the new nodes, there will continue to be 3 master nodes.

Yes, see these docs:

After a node joins or leaves the cluster, Elasticsearch reacts by automatically making corresponding changes to the voting configuration in order to ensure that the cluster is as resilient as possible.

@artechkey

Also, is there a way to allocate all roles except "master" to a node other than explicitly specifying all the roles in the yml file?

No, see these docs:

You define a node’s roles by setting node.roles in elasticsearch.yml . If you set node.roles , the node is only assigned the roles you specify. If you don’t set node.roles , the node is assigned the following roles:

[artechkey]

Thank you David.

After a node joins or leaves the cluster, Elasticsearch reacts by automatically making corresponding changes to the voting configuration in order to ensure that the cluster is as resilient as possible.

This is my understanding as well. But the docs also state the below:

Also, there are situations where the most resilient configuration includes unavailable nodes or does not include some available nodes. In these situations, the voting configuration differs from the set of available master-eligible nodes in the cluster. In these situations, the voting configuration differs from the set of available master-eligible nodes in the cluster.

Assuming that cluster.auto_shrink_voting_configuration is set to true, then the above situation where the voting configuration contains unavailable nodes should never arise. Is this understanding correct?

Thanks,
AR.

[DavidTurner]

@artechkey

the above situation where the voting configuration contains unavailable nodes should never arise

"never" is a little strong, it might exist as a transient situation, but ES will automatically fix things up as soon as it can

[artechkey]

Hi David,
Thank you for the confirmation.
By "never", I meant the same thing - that ES will "eventually" remove any unavailable master nodes from the voting configuration if the node is shut down as long as the property cluster.auto_shrink_voting_configuration is set to true.

Thanks,
AR.