Two Node Cluster Failover did not work

[sven_begis]

Two Node Cluster Failover did not work

Hello, I'm trying to set up a two node cluster.

• VST-ELA01
  -- RAM = 8 GB
  -- CPU = 8
  -- HD = 100 GB
  -- OS = Ubuntu 22.04 LTS

• VST-ELA02
  -- RAM = 8 GB
  -- CPU = 8
  -- HD = 100 GB
  -- OS = Ubuntu 22.04 LTS

• Elasticsearch Version = elasticsearch-7.8.1-amd64.deb

• Config VST-ELA01

cluster.name: myescluster
node.name: ${HOSTNAME}
network.host: 0.0.0.0
discovery.zen.ping.unicast.hosts: ["192.168.XXX.230", "192.168.XXX.231"]
path.data: /elastic/data
path.logs: /var/log/elasticsearch

• Config VST-ELA02

cluster.name: myescluster
node.name: ${HOSTNAME}
network.host: 0.0.0.0
discovery.zen.ping.unicast.hosts: ["192.168.XXX.230", "192.168.XXX.231"]
path.data: /elastic/data
path.logs: /var/log/elasticsearch

In Cluster mode, all works fine (All Nodes Up).

But when I shut down the first Node, VST-ELA01,
I can only GET Data from the Second Node, VST-ELA02,
if I try to Post something, I get an error

{
  "error" : {
    "root_cause" : [
      {
        "type" : "cluster_block_exception",
        "reason" : "blocked by: [SERVICE_UNAVAILABLE/2/no master];"
      }
    ],
    "type" : "cluster_block_exception",
    "reason" : "blocked by: [SERVICE_UNAVAILABLE/2/no master];"
  },
  "status" : 503
}

and in the Logs of the Second Server, VST-ELA02

[vst-ela02] master not discovered or elected yet, an election requires a node with id [kTkCnlBqQbiFpWx8QSFO4Q], 
have discovered [{vst-ela02}{5iyrLlvLTBa9xdhON3X8GQ}{mM7VG9rISdS_zJd8NsWvrw}{192.168.XXX.231}{192.168.XXX.231:9300}{dilmrt}{ml.machine_memory=8267603968,
xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] which is not a quorum; 
discovery will continue using [192.168.XXX.230:9300] from hosts providers and

What must I do that the Second Note VST-ELA02 takes the Master rolle when VST-ELA01 is down
or Elastic is stop?

I Also tried

Resilience in small clusters | Elasticsearch Guide [7.8] | Elastic

We recommend you set node.master: false on one of your two nodes so that it is not master-eligible.
This means you can be certain which of your nodes is the elected master of the cluster.
The cluster can tolerate the loss of the other master-ineligible node.
If you don’t set node.master: false on one node, both nodes are master-eligible.
This means both nodes are required for a master election.
Since the election will fail if either node is unavailable,
your cluster cannot reliably tolerate the loss of either node.

But has the same effect as above.

Hope someone has a solution.

Or is it not Possible what I want to do, with a Two Node Cluster setup?

As I found this in that Topic

[ES6 upgrade ES7] The two-node cluster works normally, but the master node fails, the slave node tries to reconnect, and does not recommend itself as the new master node

续上 [2019-10-30T10:36:08,674][WARN ][o.e.c.c.ClusterFormationFailureHelper] [node_20.23.72.15] master not discovered or elected yet, an election requires a node with id [pyKYCHjPQLGcM3EwI_Dc8A], have discovered [{node_20.23.72.15}{JaQ87iQpS0qLomhZvl-tPQ}{V_vcpdirTYeeRRz9xKmjlg}{20.23.72.15}{20.23.72.15:27336}{dim}] which is not a quorum; discovery will continue using from hosts providers and [{node_20.23.72.15}{JaQ87iQpS0qLomhZvl-tPQ}{V_vcpdirTYeeRRz9xKmjlg}{20.23.72.15}{20.23.72.15:27336}{dim}…

Christian_Dahlqvist
Oct 2019

You can not have a highly available cluster with 2 nodes. At least 3 master eligible nodes are required.

THX Sven

[Christian_Dahlqvist]

@sven_begis

What must I do that the Second Note VST-ELA02 takes the Master rolle when VST-ELA01 is down
or Elastic is stop?

That is not possible. You must have at least 3 master-eligible nodes in a cluster in order to be able to continue operating fully even if one of the nodes go down.

[sven_begis]

Thanks for your fast help.