Master not discovered or elected yet, an election requires 2 nodes with ids

[xziten]

Hi,

I'm trying to setup an elasticsearch Cluster. I know that this is a common topic but I would like to understand what I'm doing wrong.

Here are some informations :

OS : RHEL 7.8
ES Version : 7.10

Nodes :

• 3 Masters / node.roles: [ data, master, voting_only ]
• 1 Data Hot / node.roles: [ data_hot ]
• 2 Data Warm / node.roles: [ data_warm ]
• 3 Data Cold / node.roles: [ data_cold ]

Step by step :

1. Configuring master Nodes

cluster.name: my-cluster
node.name: my-master-1(2)(3)
node.roles: [ data, master, voting_only ]
network.host: 0.0.0.0
discovery.seed_hosts:
- 127.0.0.1
- 192.168.1.1
- 192.168.1.2
- 192.168.1.3
- 192.168.1.4
- 192.168.1.5
- 192.168.1.6
- 192.168.1.7
- 192.168.1.8
cluster.initial_master_nodes: ["my-master-1", "my-master-2", "my-master-3"]

2. Starting

When I started, I did some mistakes in the configuraton so I had to relaunch 2 of the nodes.

Then, impossible to do a /_cluster/health and got this in the log :

[TIMESTPAMP][WARN ][o.e.c.c.ClusterFormationFailureHelper] [my-master-1] master not discovered or elected yet, an election requires 2 nodes with ids [F7cuZHd3T4m-Hor0AY4AGQ, eu2wjtIlSv6WF4yOFAW8Iw], have discovered [{my-master-1]}{eu2wjtIlSv6WF4yOFAW8Iw}{xrvNSaydSjWOHmYdBJcfmw}{192.168.1.1}{192.168.1.1:9300}{dmv}{rack=r1, xpack.installed=true, transform.node=false}, {my-master-2}{F7cuZHd3T4m-Hor0AY4AGQ}{GrL4NArxQ_yqh7ICI7TFHQ}{192.168.1.2}{192.168.1.2:9300}{dmv}{rack=r1, xpack.installed=true, transform.node=false}, {my-master-3}{3Wu1yarjR1yIqltD8Gh1fw}{IoGq1vesTUOTafSjBcVfGg}{192.168.1.3}{192.168.1.3:9300}{dmv}{rack=r1, xpack.installed=true, transform.node=false}] which is not a quorum; discovery will continue using [127.0.0.1:9300, 192.168.1.2:9300, 192.168.1.3:9300, 192.168.1.4:9300, 192.168.1.5:9300, 192.168.1.6:9300, 192.168.1.7:9300, 192.168.1.8:9300, 192.168.1.9:9300] from hosts providers and [{my-master-1}{eu2wjtIlSv6WF4yOFAW8Iw}{xrvNSaydSjWOHmYdBJcfmw}{192.168.1.1}{192.168.1.1:9300}{dmv}{rack=r1, xpack.installed=true, transform.node=false}] from last-known cluster state; node term 0, last-accepted version 0 in term 0

1. Do you have any idea of the problem ?
2. How to fix without reinstall the nodes ?

I tried to restart the nodes but it doesn't fix anything

Thank you !

[Christian_Dahlqvist]

All master nodes should not be voting_only as none of these will actually be able to become master. The voting_only setting is for cases where you have an arbiter deployed and you should never have more than one voting_only node in a cluster.