Voting Only Master Node


(Matt Meredith) #1

Bringing this github conversation to this forum: https://github.com/elastic/elasticsearch/issues/14340

What is the best way to create a Master Node that can vote in a new Master Election, but itself will never get elected to be the cluster Master.
Option 1: Create a Zen Voting Only role
Option 2: Create a Zen weighted point system - where electable nodes get more points in the election

I realize this may not be possible today, but any ideas in carrying this suggestion into the roadmap would be great.

Problem:
The standard HA configuration of 3 Master nodes, will tolerate the failure of 1 master node.

2 Master nodes are required to achieve a quorum to elect a new master (and to avoid split-brain):

discovery.zen.minimum_master_nodes: 2

This requires 3 fault tolerant Zones...
If you only build in 2 fault tolerant Zones, you cannot distribute the master nodes sufficiently:
Zone A - Master1
Zone B - Master2, Master3

If you lose Zone A, you are cool. Master2 and Master3 will form a quorum and elect one of them as the master

If you lose Zone B, you are in trouble. Master1 will not have enough nodes to form a quorum. You can force the minimum_master_nodes to 1 and bootstrap Master1, but if Zone B's service is restored you will be in a split-brain status.

Business Justification:
In AWS, most regions offer a third Availability Zone, which certainly helps solve this problem. Because AWS charges for cross-AZ bandwidth, it means that the billable bandwidth can double, as all data nodes could be working with a cross-AZ master node. Today, only 50% of the nodes communicate across-AZ (as the rest talk to the master locally).

Others have also suggested further cost savings by utilizing a smaller class hardware for the Voting-only node, as it will never need to aggregate requests.

Just some ideas, would love to hear your thoughts.


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.