Node not able to be a master node in case elastic search service stopped on another master node

Elastic Stack Elasticsearch
1

For a poc purpose we have created a 2 node Elasticsearch cluster on windows machines. Cluster is working fine when both nodes are running.
To test failure scenario we stopped elasticSearch on one machine we are expecting other node to take role of master node and serve all the requests as both nodes are configured as master. But it is not working instead it is giving error of 'master-not-discovered-exception-from-node' when checking health point of node.

after enabling trace information on node below error was observed
[2024-07-17T06:59:54,190][WARN ][o.e.c.c.ClusterFormationFailureHelper] [uwperfapp000003] master not discovered or elected yet, an election requires a node with id [Buas8feHSYePYsRWH6hxnA], have only discovered non-quorum [{uwperfapp000003}{oh3rBmLdRhqJD-aQkqSwuw}{88XTjGJTT_ywKSTJp8fLrA}{10.0.3.7}{10.0.3.7:9300}{cdfhilmrstw}]; discovery will continue using [10.0.3.13:9300] from hosts providers and [{uwperfapp000003}{oh3rBmLdRhqJD-aQkqSwuw}{88XTjGJTT_ywKSTJp8fLrA}{10.0.3.7}{10.0.3.7:9300}{cdfhilmrstw}] from last-known cluster state; node term 5, last-accepted version 203 in term 5

Note - service stopped on node with id Buas8feHSYePYsRWH6hxnA

2

Elasticsearch always need a majority of master eligible nodes to be available in order to elect a master so a 2-node cluster can never be highly available as both nodes are required to form a majority. If you want to be able to have a fully functional cluster even when you lose one node you need at least 3 master eligible nodes on separate hardware. You can read more about this in the documentation.

1 Like