Elasticsearch Master Quorum is lost

Elastic Stack Elasticsearch
1

Hello :

We are having Elasticsearch version 7.8.0 , running with 6 node cluster.

All 6 nodes were data and master nodes.

We have lost 3 Nodes out of 6 , and now we have message in logfile

[2023-02-14T10:58:47,994][WARN ][o.e.c.c.ClusterFormationFailureHelper] [node-3] master not discovered or elected yet, an election requires at least 3 nodes with ids from [mD9UOfWDR_SL8XOsnvyCNQ, anO0mFz5Qgui3pSGiJI2VQ, jTop1VSZQMOcQBcR8vag2g, -8Oash5YSu6Jc8HWdzzthg, YNoy2TZDQ96amRr5prh8yw], have discovered [{node-3}{nkyEDQQVQieuz2Gn7Y6nMQ}{mef9dxGfRVOANdBBiP5tiQ}{172.24.22.221}{172.24.22.221:9300}{dilmrt}{ml.machine_memory=33511661568, rack=r1, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, {node-rh-1}{YNoy2TZDQ96amRr5prh8yw}{IMlQXxgHRAOynU_OinrNCw}{172.24.22.183}{172.24.22.183:9300}{dilmrt}{ml.machine_memory=33511661568, rack=r1, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}, {node-rh-2}{jTop1VSZQMOcQBcR8vag2g}{u13k64eHSoaodxEGbVegcg}{172.24.22.126}{172.24.22.126:9300}{dilmrt}{ml.machine_memory=33511661568, rack=r1, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}] which is not a quorum; discovery will continue using [172.24.22.183:9300, 172.24.22.126:9300] from hosts providers and [{node-3}{nkyEDQQVQieuz2Gn7Y6nMQ}{mef9dxGfRVOANdBBiP5tiQ}{172.24.22.221}{172.24.22.221:9300}{dilmrt}{ml.machine_memory=33511661568, rack=r1, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 10, last-accepted version 3258 in term 10

ELASTICSEARCH is looking for Nodes with ID's
[mD9UOfWDR_SL8XOsnvyCNQ, anO0mFz5Qgui3pSGiJI2VQ, jTop1VSZQMOcQBcR8vag2g, -8Oash5YSu6Jc8HWdzzthg, YNoy2TZDQ96amRr5prh8yw]

We are having only below nodes :

[node-rh-2], node ID [jTop1VSZQMOcQBcR8vag2g],
[node-rh-3], node ID [nkyEDQQVQieuz2Gn7Y6nMQ]
[node-rh-1], node ID [YNoy2TZDQ96amRr5prh8yw]

Hence would like your help , if you could help me how to fix this forcefully and make up cluster.

  1. How to fix the quorum with available nodes
  2. How to fix UUID for a node during startup so that , atleast my cluster would start.
  3. I do not have BACKUP of my data.
2

Elasticsearch version 7.8 is EOL and no longer supported. Please upgrade ASAP.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

3

My Elasticsearch YML FILE :

cluster.name: PD-JC-ES-ADVSRCH-C02
node.name: node-rh-2
node.attr.rack: r1
path.data: /usrdata/elastic/data
path.logs: /usrdata/elastic/logs
bootstrap.memory_lock: true
network.host: 172.24.22.126
http.port: 9207
discovery.seed_hosts: ["172.24.22.183", "172.24.22.126", "172.24.22.221"]
action.auto_create_index: "+.*"
prometheus.indices: true
thread_pool.write.queue_size: 60000
indices.breaker.request.limit: 0.5%
indices.breaker.fielddata.limit: 1%
indices.breaker.accounting.limit: 1%
indices.fielddata.cache.size: 5%
4

Okay , we have fixed the issue by following the document :

Thank You

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.