I am new to Elastic search and trying to understand what is the minimum requirement for a ES cluster keeping availability in mind.
Note that I am considering all the nodes are master-eligible and data node and running Elastic-Search -7.8.
I started with minimum 2 nodes setup for this. Since we have master and data nodes here. By default all nodes are master-eligible unless otherwise specified. Say node 1 is master and node 2 is data (master-eligible). Say node-1 is down , then the only available node is 2 and it becomes master. But what if there is a network break between these nodes? Then node-1 will still be master but node-2 thinks node-1 is down and it elects itself as a master. When the connectivity is restored we have 2 masters for a cluster.
So next option was to have 3 nodes : In this case say node-1 is the master and it is down . We have node-2 and node-3 which have the right to vote for master elections and it may happen that node-2 elects itself for master and node-3 does the same. Now the cluster does not have a master.
The 4 node theory seemed slightly better than others. A master node going down would cause 3 nodes in the cluster to be up and in that use-case we will not face split brain. Having said that if 2 nodes goes down , we will still face the same , but the chances are less.
The question is regarding the statement with 3 node setup where there is no majority on voting. Is this really possible or ES is intelligent enough to come to a conclusion and have a master node?