Which is the main master in 3 dedicated master node?

Elastic Stack Elasticsearch
1

Suppose I have a 3 dedicated master node and 4 data nodes. At a moment are all 3 acting as a master or is there a supreme master between them so that when it goes down another is elected as Supreme Master ?

Are 3 master acting together at one time with each controlling some primary shards? So when one goes down, another master will take its primary shards and that will be done by voting?

2

One of the nodes will be acting as master at any time with the others monitoring and waiting to step in if required.

3

Thanks for the reply. But then what is the need to make a 3 node cluster. I can also make a 2 node cluster right, with one of my data nodes as voting node.

Also can you point me to an article where these things are explained?

4

You can make 3 if your data nodes master eligible and skip dedicated master nodes altogether and the cluster will still work. Having dedicated master nodes is usually recommended for large clusters and/or very busy clusters where e.g. load may lead to long GC or the current master node not having enough resources to complete tasks in a timely fashion.

5

See for instance Resilience in small clusters | Elasticsearch Guide [8.11] | Elastic and the surrounding pages in the manual.

6

Hi, suppose I have 2 dedicated master and 3 data nodes. One of the data nodes is also voting eligible.
Suppose m1 is active master.
If m1 and m2 loses communication, will m2 elect itself as master since data node can vote in master election process. Will this cause split brain?

7

You should ALWAYS look to have at least three master eligible nodes in a cluster, never two.

8

Yes you are correct but I want to understand the reasons. Will voting only data node help in avoiding splitbrain if there are 2 dedicated masters?

M1 is active. Suppose m1 and m2 goes lose communication between them. How would the master election process proceed from here? M1 is already master and has connections to all data nodes. M2 thinks other master is dead so will try to elect itself as master right. So will the data node vote it and make it a second active master hence causing splitbrain Or voting only data nodes gets to vote only in case of tiebreaker?

9

The third node will determine which of the other nodes that is elected. Only one can succeed.

10

Yes, M2 will think M1 has failed but none of the other nodes will agree, so it won't be elected.

11

Thanks a lot. That's what I needed.

So though not ideal but we can have 2 dedicated master node with one of the data node as voting only correct? This won't cause split brain?

12

Indeed, you're safe from split-brain inconsistencies no matter what your configuration. Note that the manual suggests exactly this configuration (emphasis mine):

You may configure one of your master-eligible nodes to be a voting-only node so that it can never be elected as the master node. For instance, you may have two dedicated master nodes and a third node that is both a data node and a voting-only master-eligible node. This third voting-only node will act as a tiebreaker in master elections but will never become the master itself.

1 Like
13

Thanks. The terminology 'master eligible' for voting only node is very bad.

14

This too is documented:

It may seem confusing to use the term "master-eligible" to describe a voting-only node since such a node is not actually eligible to become the master at all. This terminology is an unfortunate consequence of history: master-eligible nodes are those nodes that participate in elections and perform certain tasks during cluster state publications, and voting-only nodes have the same responsibilities even if they can never become the elected master.

15

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.