# Nodes not electing mater when master goes down **URL:** \ **Category:** Elasticsearch\ **Created:** [November 30, 2024, 10:31am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257 "2024-11-30T10:31:52Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Wajahat\_Iqbal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wajahat_iqbal/32/139609_2.png) [@Wajahat\_Iqbal](https://discuss.elastic.co/u/Wajahat_Iqbal)\ **Post date:** [November 30, 2024, 10:31am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/1 "2024-11-30T10:31:52Z")
ERROR THIS `[2024-11-30T10:19:13,234][WARN][o.e.c.c.ClusterFormationFailureHelper] [node-3] master not discovered yet: have discovered [{node-3}{CpTTLncQTpq2jmzKCp6BfQ}{OdGCenkKT52GKc75CphmxQ}{10.0.0.6}{10.0.0.6:9300}{cdhilrstw}{ml.machine_memory=8271114240, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, {node-1}{b3lPF74ZTO-wg9eIjgTuSA}{ZjQP7vf7TOSFTLiwiQzC-Q}{10.0.0.4}{10.0.0.4:9300}{cdhilmrstw}{ml.machine_memory=8271114240, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}]; discovery will continue using [10.0.0.4:9300, 10.0.0.5:9300] from hosts providers and [{node-2}{Id07aq88SF-nclMGG-sOww}{PbX19mFtSxWwXCM9MMCs9w}{10.0.0.5}{10.0.0.5:9300}{cdhilmrstw}{ml.machine_memory=8271110144, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}, {node-1}{b3lPF74ZTO-wg9eIjgTuSA}{ZjQP7vf7TOSFTLiwiQzC-Q}{10.0.0.4}{10.0.0.4:9300}{cdhilmrstw}{ml.machine_memory=8271114240, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}] from last-known cluster state; node term 35, last-accepted version 384 in term 35` node-1 is master eligible node-2 is master node-2 is not master when i turn off , the expected behaviour should be to elect a new master , node-1 but this is not happening this is my yml file ```auto cluster.name: markaz-es-dev node.name: node-1 node.master: true network.host: 10.0.0.4 http.port: 9200 discovery.zen.minimum_master_nodes: 2 discovery.seed_hosts: ["10.0.0.4","10.0.0.5","10.0.0.6"] cluster.initial_master_nodes: ["node-1","node-2","node-3"] azure.client.default.timeout: 3s azure.client.default.max_retries: 2 azure.client.default.endpoint_suffix: core.windows.net logger.org.elasticsearch.repositories.azure: DEBUG xpack.security.enabled: false path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch ``` ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [November 30, 2024, 7:07pm UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/2 "2024-11-30T19:07:57Z")
Which version of Elasticsearch are you using? It must be quite old if the configuration settings are correct. I would recommend that you upgrade to the latest version. > [@Wajahat\_Iqbal](#): > > node-1 is master eligible > node-2 is master > node-2 is not master I am not sure I follow this. Do you have 3 nodes that are all master eligible (that is what it looks like based on the config)? ---
**Author:** ![Wajahat\_Iqbal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wajahat_iqbal/32/139609_2.png) [@Wajahat\_Iqbal](https://discuss.elastic.co/u/Wajahat_Iqbal)\ **Post date:** [December 2, 2024, 4:38am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/3 "2024-12-02T04:38:17Z")
I am using 7.10.0, Initially I had 2 nodes, Both were master eligible, than I learned if you have 2 nodes and your master goes down, the other node cannot become master becuase of you need 2 nodes to elect a master, I added another node made it master eligible, with same configuration as above. Still, I was facing same problem. than i made node-3 non-master node to check still same problem. i reverted the setting and made node-3 master node. Now everything is working fine. Now I am confues. I guess what happen was, when I created 2 Node, and created the cluster only 2 node were participating in election, even when I added the node-3 , until i restarded the whole clustor. Can you please share any guidelines, Becuase this was an experiment, i need to recreate this in production. with high traffic we are migrating or atlas search index to self-hosted Elasticsearch. i need to make it resilient, I have also added the snapshot. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [December 2, 2024, 5:47am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/4 "2024-12-02T05:47:37Z")
> [@Wajahat\_Iqbal](#): > > I am using 7.10.0 Why are you using such an old version that has been EOL a long time? > [@Wajahat\_Iqbal](#): > > Both were master eligible, than I learned if you have 2 nodes and your master goes down, the other node cannot become master becuase of you need 2 nodes to elect a master, I added another node made it master eligible, with same configuration as above. Still, I was facing same problem. That does not make sense. Did you verify that all 3 nodes were part of the cluster after you added the new node, e.g. through the [cat nodes API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-nodes.html)? What does the API show? Did the logs of the new node indicate that it formed up with the cluster? > [@Wajahat\_Iqbal](#): > > `discovery.zen.minimum_master_nodes: 2` This setting is not valid or needed in Elasticsearch 7.0+. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [December 2, 2024, 8:25am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/5 "2024-12-02T08:25:11Z")
The issue is that `node-3` is not master-eligible. > [@Wajahat\_Iqbal](#): > > `have discovered [{node-3}{...}{cdhilrstw}` NB the lack of `m` in `cdhilrstw`. ---
**Author:** ![Wajahat\_Iqbal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wajahat_iqbal/32/139609_2.png) [@Wajahat\_Iqbal](https://discuss.elastic.co/u/Wajahat_Iqbal)\ **Post date:** [December 2, 2024, 11:05am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/6 "2024-12-02T11:05:03Z")
When node-3 was master node I was till facing problem ---
**Author:** ![Wajahat\_Iqbal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wajahat_iqbal/32/139609_2.png) [@Wajahat\_Iqbal](https://discuss.elastic.co/u/Wajahat_Iqbal)\ **Post date:** [December 2, 2024, 11:06am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/7 "2024-12-02T11:06:25Z")
I was using 7.10 as an experiment on production I'll use 8+. Yes did check all of my nodes were master. node-1 was master , node-2 and node-3 was master eligible. Anyway I am reconfiguring it for Production with 8, as this was an experiment to check if we can migrate from atlas search to Elasticsearch. if you guys have any suggestion please let me know ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [December 2, 2024, 11:42am UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/8 "2024-12-02T11:42:52Z")
> [@Wajahat\_Iqbal](#): > > When node-3 was master node I was till facing problem That may be true, but it was not the _same_ problem. The logs you shared are unambiguous: `node-3` was not master-eligible. ---
**Author:** ![Wajahat\_Iqbal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wajahat_iqbal/32/139609_2.png) [@Wajahat\_Iqbal](https://discuss.elastic.co/u/Wajahat_Iqbal)\ **Post date:** [December 2, 2024, 12:04pm UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/9 "2024-12-02T12:04:04Z")
Yes,these logs were taken when node-3 was removed from master. after that i re configure it to be master, then the clustor start working , and problem was fixed ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [December 2, 2024, 1:57pm UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/10 "2024-12-02T13:57:43Z")
> [@Wajahat\_Iqbal](#): > > after that i re configure it to be master, then the clustor start working , and problem was fixed Then Elasticsearch is working as designed, if you had only 2 master-eligible nodes in the cluster and removed one, you won't have a working cluster as you would need at least 3 master eligible nodes. I recommend that you read this documentation about [resilience](https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability-cluster-design.html) and this one about [resilience in smaller clusters](https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability-cluster-small-clusters.html). ---
**Author:** ![Wajahat\_Iqbal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wajahat_iqbal/32/139609_2.png) [@Wajahat\_Iqbal](https://discuss.elastic.co/u/Wajahat_Iqbal)\ **Post date:** [December 2, 2024, 4:35pm UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/11 "2024-12-02T16:35:21Z")
Thanks everyone, I just setup the clustor for production with version 8.16.1 and every thing is working perfectly fine. ---
**Author:** ![Wajahat\_Iqbal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wajahat_iqbal/32/139609_2.png) [@Wajahat\_Iqbal](https://discuss.elastic.co/u/Wajahat_Iqbal)\ **Post date:** [December 2, 2024, 6:26pm UTC](https://discuss.elastic.co/t/nodes-not-electing-mater-when-master-goes-down/371257/12 "2024-12-02T18:26:54Z")
But now I am having trouble connecting to elasticsearch clustor using loadbalancer, I was using azure load balancer also tried with nginx