Master-Eligible node cluster resiliency for ElasticSearch 7.x

Unfortunately, the testing failed - let me walk through the process.

1 - initially I setup one dedicated master and one dedicate data node using node.roles={master|data} property and also having the cluster.initial_master_nodes set for the one master node.

2 - after step-1, I have a successful cluster formed.

curl http://escluster.local.internal:9200/_cat/nodes?v
ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.192.106.236           29          32   0    0.04    0.03     0.01 m         *      ip-10-192-106-236
10.192.104.252           50          33   0    0.01    0.07     0.05 dilrt     -      ip-10-192-104-252

3 - then I removed this cluster.initial_master_nodes property from elasticearch.yml file of existing nodes and also ensured that new launched systems do not have this property configured.

4 - added one more master node in the pool and cluster was still healthy with one more master in the mix.

curl http://escluster.local.internal:9200/_cat/nodes?v
ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
10.192.106.236           29          32   0    0.04    0.03     0.01 m         *      ip-10-192-106-236
10.192.105.242           19          27   9    0.90    0.60     0.25 m         -      ip-10-192-105-242
10.192.104.252           50          33   0    0.01    0.07     0.05 dilrt     -      ip-10-192-104-252

5 - at this stage, ip-10-192-106-236 is still the active master node.

6 - I terminate that node ip-10-192-106-236 assuming that ip-10-192-105-242 will become newly active master node. However, the cluster state went bad again - with following error,

curl http://escluster.local.internal:9200/_cat/nodes?v
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503} 

Any thoughts on this?