# Splitbrain on a 4 node implementation **URL:** \ **Category:** Elasticsearch\ **Created:** [January 22, 2018, 9:56am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461 "2018-01-22T09:56:47Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Edmund\_Cheng](https://avatars.discourse-cdn.com/v4/letter/e/fbc32d/32.png) [@Edmund\_Cheng](https://discuss.elastic.co/u/Edmund_Cheng)\ **Post date:** [January 22, 2018, 9:56am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/1 "2018-01-22T09:56:47Z")
Hi, I have done some reading about the splitbrain scenario, but seems to have hit a dilemma. e.g for this structure: machine1: 2 ES nodes (with rack id 1) machine2: 2 ES nodes (with rack id 2) In the above case, all the settings are default, except for awareness, which i wanted the pri/replica shards to be distributed across racks. So if there is a network issue between the 2 machines, I will have a split brain scenario since min. master is 1. 1. I can set min. master to 3 to avoid split brain: - but ES will be inoperable if network issue happens or even - if 1 machine goes down, the ES is also inoperable, thus losing HA 2. if I set min.master to anything less than 3, - I will get split brain if there is network issue between the 2 machines - but if 1 machine goes down, I can still have 1 machine which is working.(HA) I am not sure this is a right design or I am missing some better configs. Any advise will be great! ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 22, 2018, 9:59am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/2 "2018-01-22T09:59:46Z")
You should set minimum\_master\_nodes to 3, as this will allow 1 node to go down. In order to build a HA cluster where a machine can be allowed to fail while still allow the cluster to be fully operational, a minimum of 3 machines is required (this can however be much smaller as it generally only need to host a single, dedicated master node). ---
**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\ **Post date:** [January 22, 2018, 10:09am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/3 "2018-01-22T10:09:56Z")
FYI. [Zen Discovery | Elasticsearch Reference [6.0] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/6.0/modules-discovery-zen.html#master-election) ```bash # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1): # discovery.zen.minimum_master_nodes: 3 ``` total number of master-eligible nodes mean all es nodes have below config. > node.master: true ---
**Author:** ![Edmund\_Cheng](https://avatars.discourse-cdn.com/v4/letter/e/fbc32d/32.png) [@Edmund\_Cheng](https://discuss.elastic.co/u/Edmund_Cheng)\ **Post date:** [January 22, 2018, 10:56am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/4 "2018-01-22T10:56:38Z")
Thanks Christian, "You should set minimum\_master\_nodes to 3, as this will allow 1 node to go down. " Yes this is the best it can go i guess, because if one more node goes down, the ES will be inoperable since min.master of 3 is not met. _"In order to build a HA cluster where a machine can be allowed to fail while still allow the cluster to be fully operational, a minimum of 3 machines is required (this can however be much smaller as it generally only need to host a single, dedicated master node)."_ So basically, for these 2 machines with 2 nodes, its not really an ideal implementation if we need HA and avoid split brain at the same time? what do you mean by (much smaller)? ---
**Author:** ![Edmund\_Cheng](https://avatars.discourse-cdn.com/v4/letter/e/fbc32d/32.png) [@Edmund\_Cheng](https://discuss.elastic.co/u/Edmund_Cheng)\ **Post date:** [January 22, 2018, 10:59am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/5 "2018-01-22T10:59:28Z")
Hi, yes i understand that variable, but I will lose the ES if one machine goes down since min.master is not met. Avoided split brain, but causes the ES to stop working. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 22, 2018, 11:11am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/6 "2018-01-22T11:11:56Z")
> [@Edmund\_Cheng](#): > > So basically, for these 2 machines with 2 nodes, its not really an ideal implementation if we need HA and avoid split brain at the same time? what do you mean by (much smaller)? Correct. For a small cluster with a reasonably sized cluster state, 1-2 CPU cores and 2-4GB of heap for a dedicated master node is often enough. ---
**Author:** ![Edmund\_Cheng](https://avatars.discourse-cdn.com/v4/letter/e/fbc32d/32.png) [@Edmund\_Cheng](https://discuss.elastic.co/u/Edmund_Cheng)\ **Post date:** [January 22, 2018, 11:38am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/7 "2018-01-22T11:38:11Z")
hi, I see.. thanks.. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 19, 2018, 11:38am UTC](https://discuss.elastic.co/t/splitbrain-on-a-4-node-implementation/116461/8 "2018-02-19T11:38:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.