# Proper way to force a Master Election after Split Brain occurs **URL:** \ **Category:** Elasticsearch\ **Created:** [October 26, 2015, 8:14pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022 "2015-10-26T20:14:23Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Markus\_Gibson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markus_gibson/32/5498_2.png) [@Markus\_Gibson](https://discuss.elastic.co/u/Markus_Gibson)\ **Post date:** [October 26, 2015, 8:14pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/1 "2015-10-26T20:14:23Z")
I would like to know the proper way to correctly handle a split brain situation, I am constantly seeing only "failed to send join request to master" messages in an attempt to connect to a node that is no longer there. The cluster is dead in the water, and does not respond to any requests. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 27, 2015, 1:42am UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/2 "2015-10-27T01:42:35Z")
Do you not have minimum masters set? Have you restarted the existing/old master? What does `_cat/master` show for each node in the cluster? ---
**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\ **Post date:** [October 27, 2015, 1:49am UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/3 "2015-10-27T01:49:01Z")
How many nodes cluster you have and how do you determine it as split brain? Does all the nodes work independently and doesn't join cluster. What is the way of your node cluster join mechanism. (Zen or cloud plugins like cloud-aws) -- Niraj ---
**Author:** ![Markus\_Gibson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markus_gibson/32/5498_2.png) [@Markus\_Gibson](https://discuss.elastic.co/u/Markus_Gibson)\ **Post date:** [October 27, 2015, 1:56pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/4 "2015-10-27T13:56:54Z")
Hi, i have the following setting: ``` discovery.zen.minimum_master_nodes: 1 ``` The old master in this case is completely gone. ---
**Author:** ![Markus\_Gibson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markus_gibson/32/5498_2.png) [@Markus\_Gibson](https://discuss.elastic.co/u/Markus_Gibson)\ **Post date:** [October 27, 2015, 2:03pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/5 "2015-10-27T14:03:51Z")
1. I'm thinking that this is split brain, since none of the masters will actually hold an election as they are still looking for the old master. 2. We have 30 nodes total 3 - master, 2 - client, 25 - data nodes. 3. No nodes work independently We also have the following set cloud.node.auto\_attributes: true However we are not in AWS, but an internal cloud solution similar to VMware. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 27, 2015, 3:28pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/6 "2015-10-27T15:28:14Z")
> [@Markus\_Gibson](#): > > discovery.zen.minimum\_master\_nodes: 1 If you have 3 master eligible nodes, you should have discovery.zen.minimum\_master\_nodes set to 2, not 1, as described in [the Definitive Guide](https://www.elastic.co/guide/en/elasticsearch/guide/current/_important_configuration_changes.html#_minimum_master_nodes). ---
**Author:** ![Markus\_Gibson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markus_gibson/32/5498_2.png) [@Markus\_Gibson](https://discuss.elastic.co/u/Markus_Gibson)\ **Post date:** [October 27, 2015, 3:29pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/7 "2015-10-27T15:29:51Z")
Yes, I just found that thank you. We use Chef to write out the config, now to figure out why chef suddenly started writing a 1 rather than a 2 ---
**Author:** ![Markus\_Gibson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/markus_gibson/32/5498_2.png) [@Markus\_Gibson](https://discuss.elastic.co/u/Markus_Gibson)\ **Post date:** [October 27, 2015, 3:52pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/8 "2015-10-27T15:52:32Z")
I've changed it to 2 however, it's still looking for an old master node. Is there a cache somewhere I need to clear? ---
**Author:** ![niraj\_kumar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niraj_kumar/32/4130_2.png) [@niraj\_kumar](https://discuss.elastic.co/u/niraj_kumar)\ **Post date:** [October 27, 2015, 4:13pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/9 "2015-10-27T16:13:57Z")
I believe there is no cache mechanism. You can do a quick reboot i believe taking a good amount of downtime. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:42pm UTC](https://discuss.elastic.co/t/proper-way-to-force-a-master-election-after-split-brain-occurs/33022/10 "2017-07-05T23:42:23Z")