Elasticsearch 8.8: Master not discovered or elected yet, an election requires at least 2 nodes with ids from [..]

Elastic Stack Elasticsearch
1

I am creating a multinode cluster (3 Master Nodes), having the configuration like

xpack.ml.enabled: false
xpack.security.enabled: false
network.host: [_local_, _site_]
path.data: /data/esdata
path.logs: /data/logs
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12

cluster.name: XXX-es-prod
node.name: XXX-es-prod-m3
node.roles: [ "master"]
cluster.initial_master_nodes:
        - XXX-es-prod-m1
discovery.seed_hosts:
 - 172.XX.XX:9300
 - 172.XX.XX:9300

search.allow_expensive_queries: false

When I start elasticsearch in all the 3 nodes, I am getting this error

[2023-07-10T15:10:26,723][WARN ][o.e.c.c.ClusterFormationFailureHelper] [XXX-es-prod-m3] master not discovered or elected yet, an election requires at least 2 nodes with ids from [T3Sqp5utTkaZRpgGB4fJbw, grN9lbg0QJ2zEXQcsawfQg, mE4YCC9qSX-81b2NzbHZIA], have only discovered non-quorum [{XXX-es-prod-m3}{T3Sqp5utTkaZRpgGB4fJbw}{qdzooCwxRBezB_4tmqeyZw}{XXX-es-prod-m3}{172.XX.XX.X}{172.XX.XX.X:9300}{m}{8.8.2}, {XXX-es-prod-m2}{T3Sqp5utTkaZRpgGB4fJbw}{ShqjmPKBT4aDQXMxpxgzQA}{XXX-es-prod-m2}{172.XX.XX.X}{172.XX.XX.X:9300}{m}{8.8.2}, {XXX-es-prod-m1}{T3Sqp5utTkaZRpgGB4fJbw}{l_xowlm9R0GTxehVtgT22w}{XXX-es-prod-m1}{172.XX.XX.X}{172.XX.XX.X:9300}{m}{8.8.2}]; discovery will continue using [172.XX.XX.X:9300, 172.XX.XX.X:9300] from hosts providers and [{XXX-es-prod-m3}{T3Sqp5utTkaZRpgGB4fJbw}{qdzooCwxRBezB_4tmqeyZw}{XXX-es-prod-m3}{172.XX.XX.X}{172.XX.XX.X:9300}{m}{8.8.2}] from last-known cluster state; node term 4, last-accepted version 61 in term 4; for troubleshooting guidance, see Troubleshooting discovery | Elasticsearch Guide [8.8] | Elastic

Can anyone please help me resolve this issue ?

2

Hi,
At first sight, it looks like a cluster has already formed, and it does not match in some way with your current configuration.
Please read Bootstrapping a cluster | Elasticsearch Guide [8.8] | Elastic , it clarifies cluster formation mechanisms.
In your case (assuming you don't have any valuable indexed data or cluster assets), I would

  • remove path.data contents on each node
  • add all 3 nodes names in cluster.initial_master_nodes
  • restart your cluster

And also make sure that you have at least one data node (not clear from you post)!

3

Thanks @vincenbr ! After removing path.data content on each node and restarting the cluster, the issue got fixed.