# Master fall
**URL:** https://discuss.elastic.co/t/master-fall/10799
**Category:** Elasticsearch
**Created:** [February 19, 2013, 12:59pm UTC](https://discuss.elastic.co/t/master-fall/10799 "2013-02-19T12:59:53Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)
#### Post date: [February 19, 2013, 12:59pm UTC](https://discuss.elastic.co/t/master-fall/10799/1 "2013-02-19T12:59:53Z")
Hi,
What's the best practice for an elasticsearch client to handle a scenario
in which elasticsearch master falls?
From my tests I can connect to any of the slaves and continue on. Does it
matter that the client connect to a slave and not to a master?
Thanks,
Ophir
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@polyfractal](https://discuss.elastic.co/u/polyfractal)
#### Post date: [February 19, 2013, 1:17pm UTC](https://discuss.elastic.co/t/master-fall/10799/2 "2013-02-19T13:17:14Z")
Elasticsearch will automatically elect a new master if the old one becomes
unavailable, but master's are not necessary for search/indexing operations.
E.g. any node in the cluster is fully capable of servicing a search
request, not just the master node.
The master is in charge of managing cluster metadata and state (mappings,
new indices, etc), in addition to normal indexing/search requirements.
It is also important to have your minimum\_master\_nodes setting configured
properly, to help avoid split-brains.
-Zach
On Tuesday, February 19, 2013 7:59:53 AM UTC-5, Ophir Michaeli wrote:
> Hi,
>
> What's the best practice for an elasticsearch client to handle a scenario
> in which elasticsearch master falls?
>
> From my tests I can connect to any of the slaves and continue on. Does it
> matter that the client connect to a slave and not to a master?
>
> Thanks,
> Ophir
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)
#### Post date: [February 19, 2013, 2:48pm UTC](https://discuss.elastic.co/t/master-fall/10799/3 "2013-02-19T14:48:28Z")
Hi,
It is not clear from the documentation why minimum\_master\_nodes should be
higher than 1 for more than 2 nodes, and what are the considerations to
configure properly.
Thanks
On Tuesday, February 19, 2013 3:17:14 PM UTC+2, Zachary Tong wrote:
> Elasticsearch will automatically elect a new master if the old one becomes
> unavailable, but master's are not necessary for search/indexing operations.
> E.g. any node in the cluster is fully capable of servicing a search
> request, not just the master node.
>
> The master is in charge of managing cluster metadata and state (mappings,
> new indices, etc), in addition to normal indexing/search requirements.
>
> It is also important to have your minimum\_master\_nodes setting configured
> properly, to help avoid split-brains.
>
> -Zach
>
> On Tuesday, February 19, 2013 7:59:53 AM UTC-5, Ophir Michaeli wrote:
>
> > Hi,
> >
> > What's the best practice for an elasticsearch client to handle a scenario
> > in which elasticsearch master falls?
> >
> > From my tests I can connect to any of the slaves and continue on. Does it
> > matter that the client connect to a slave and not to a master?
> >
> > Thanks,
> > Ophir
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@polyfractal](https://discuss.elastic.co/u/polyfractal)
#### Post date: [February 19, 2013, 7:06pm UTC](https://discuss.elastic.co/t/master-fall/10799/4 "2013-02-19T19:06:01Z")
The recommended value is usually (N/2 + 1) where N is the number of
master-eligible nodes.
For example, if you have min\_master\_nodes == 1 in a three node cluster, you
can easily get a "split-brain". Imagine one node drops out of the cluster,
but remains active (e.g. the server didn't die, but the network connection
between it and the rest of the nodes died). This solitary node sees the
other two departing and thinks it is the only survivor, so it elects itself
as master and forms a new cluster.
Meanwhile, the other two nodes see the single node leave and just carry on
as usual.
Now, when the network disconnect resolves itself, the third node does not
attempt to rejoin the cluster because it thinks _it is_ the cluster and the
other two are missing. This is a split-brain, where you have two clusters
when only one should exist. Even worse, they don't realize there are two
clusters so now you have rapidly diverging data states because they aren't
trying to remerge.
Min\_master\_nodes prevents this. If min\_master\_nodes == 2 in the above
scenario, the solitary node sees only one master eligible node (itself),
which is less than the required 2 nodes. Therefore, instead of electing
itself as master, it simply keeps pinging the old nodes until the network
disconnect resolves, then rejoins.
-Zach
On Tuesday, February 19, 2013 9:48:28 AM UTC-5, Ophir Michaeli wrote:
> Hi,
>
> It is not clear from the documentation why minimum\_master\_nodes should be
> higher than 1 for more than 2 nodes, and what are the considerations to
> configure properly.
>
> Thanks
>
> On Tuesday, February 19, 2013 3:17:14 PM UTC+2, Zachary Tong wrote:
>
> > Elasticsearch will automatically elect a new master if the old one
> > becomes unavailable, but master's are not necessary for search/indexing
> > operations. E.g. any node in the cluster is fully capable of servicing a
> > search request, not just the master node.
> >
> > The master is in charge of managing cluster metadata and state (mappings,
> > new indices, etc), in addition to normal indexing/search requirements.
> >
> > It is also important to have your minimum\_master\_nodes setting
> > configured properly, to help avoid split-brains.
> >
> > -Zach
> >
> > On Tuesday, February 19, 2013 7:59:53 AM UTC-5, Ophir Michaeli wrote:
> >
> > > Hi,
> > >
> > > What's the best practice for an elasticsearch client to handle a
> > > scenario in which elasticsearch master falls?
> > >
> > > From my tests I can connect to any of the slaves and continue on. Does
> > > it matter that the client connect to a slave and not to a master?
> > >
> > > Thanks,
> > > Ophir
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)
#### Post date: [February 20, 2013, 9:33am UTC](https://discuss.elastic.co/t/master-fall/10799/5 "2013-02-20T09:33:54Z")
Just to make sure I understood correctly, if I run 4 elasticsearch nodes (a
cluster) I'll set minimum\_master\_nodes to 3?
Because if one falls (including a case where the master falls) then the
remaining 3 can have a new master elected (they are 3 and
minimum\_master\_nodes is 3),
while the solitary node has only 1 node (itself) so it will wait till
reconnection to the others in the cluster.
On Tuesday, February 19, 2013 9:06:01 PM UTC+2, Zachary Tong wrote:
> The recommended value is usually (N/2 + 1) where N is the number of
> master-eligible nodes.
>
> For example, if you have min\_master\_nodes == 1 in a three node cluster,
> you can easily get a "split-brain". Imagine one node drops out of the
> cluster, but remains active (e.g. the server didn't die, but the network
> connection between it and the rest of the nodes died). This solitary node
> sees the other two departing and thinks it is the only survivor, so it
> elects itself as master and forms a new cluster.
>
> Meanwhile, the other two nodes see the single node leave and just carry on
> as usual.
>
> Now, when the network disconnect resolves itself, the third node does not
> attempt to rejoin the cluster because it thinks _it is_ the cluster and
> the other two are missing. This is a split-brain, where you have two
> clusters when only one should exist. Even worse, they don't realize there
> are two clusters so now you have rapidly diverging data states because they
> aren't trying to remerge.
>
> Min\_master\_nodes prevents this. If min\_master\_nodes == 2 in the above
> scenario, the solitary node sees only one master eligible node (itself),
> which is less than the required 2 nodes. Therefore, instead of electing
> itself as master, it simply keeps pinging the old nodes until the network
> disconnect resolves, then rejoins.
>
> -Zach
>
> On Tuesday, February 19, 2013 9:48:28 AM UTC-5, Ophir Michaeli wrote:
>
> > Hi,
> >
> > It is not clear from the documentation why minimum\_master\_nodes should be
> > higher than 1 for more than 2 nodes, and what are the considerations to
> > configure properly.
> >
> > Thanks
> >
> > On Tuesday, February 19, 2013 3:17:14 PM UTC+2, Zachary Tong wrote:
> >
> > > Elasticsearch will automatically elect a new master if the old one
> > > becomes unavailable, but master's are not necessary for search/indexing
> > > operations. E.g. any node in the cluster is fully capable of servicing a
> > > search request, not just the master node.
> > >
> > > The master is in charge of managing cluster metadata and state
> > > (mappings, new indices, etc), in addition to normal indexing/search
> > > requirements.
> > >
> > > It is also important to have your minimum\_master\_nodes setting
> > > configured properly, to help avoid split-brains.
> > >
> > > -Zach
> > >
> > > On Tuesday, February 19, 2013 7:59:53 AM UTC-5, Ophir Michaeli wrote:
> > >
> > > > Hi,
> > > >
> > > > What's the best practice for an elasticsearch client to handle a
> > > > scenario in which elasticsearch master falls?
> > > >
> > > > From my tests I can connect to any of the slaves and continue on. Does
> > > > it matter that the client connect to a slave and not to a master?
> > > >
> > > > Thanks,
> > > > Ophir
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@polyfractal](https://discuss.elastic.co/u/polyfractal)
#### Post date: [February 20, 2013, 11:52am UTC](https://discuss.elastic.co/t/master-fall/10799/6 "2013-02-20T11:52:43Z")
Yep, that's correct. =)
-Zach
On Wednesday, February 20, 2013 4:33:54 AM UTC-5, Ophir Michaeli wrote:
> Just to make sure I understood correctly, if I run 4 elasticsearch nodes
> (a cluster) I'll set minimum\_master\_nodes to 3?
>
> Because if one falls (including a case where the master falls) then the
> remaining 3 can have a new master elected (they are 3 and
> minimum\_master\_nodes is 3),
>
> while the solitary node has only 1 node (itself) so it will wait till
> reconnection to the others in the cluster.
>
> On Tuesday, February 19, 2013 9:06:01 PM UTC+2, Zachary Tong wrote:
>
> > The recommended value is usually (N/2 + 1) where N is the number of
> > master-eligible nodes.
> >
> > For example, if you have min\_master\_nodes == 1 in a three node cluster,
> > you can easily get a "split-brain". Imagine one node drops out of the
> > cluster, but remains active (e.g. the server didn't die, but the network
> > connection between it and the rest of the nodes died). This solitary node
> > sees the other two departing and thinks it is the only survivor, so it
> > elects itself as master and forms a new cluster.
> >
> > Meanwhile, the other two nodes see the single node leave and just carry
> > on as usual.
> >
> > Now, when the network disconnect resolves itself, the third node does not
> > attempt to rejoin the cluster because it thinks _it is_ the cluster and
> > the other two are missing. This is a split-brain, where you have two
> > clusters when only one should exist. Even worse, they don't realize there
> > are two clusters so now you have rapidly diverging data states because they
> > aren't trying to remerge.
> >
> > Min\_master\_nodes prevents this. If min\_master\_nodes == 2 in the above
> > scenario, the solitary node sees only one master eligible node (itself),
> > which is less than the required 2 nodes. Therefore, instead of electing
> > itself as master, it simply keeps pinging the old nodes until the network
> > disconnect resolves, then rejoins.
> >
> > -Zach
> >
> > On Tuesday, February 19, 2013 9:48:28 AM UTC-5, Ophir Michaeli wrote:
> >
> > > Hi,
> > >
> > > It is not clear from the documentation why minimum\_master\_nodes should
> > > be higher than 1 for more than 2 nodes, and what are the considerations to
> > > configure properly.
> > >
> > > Thanks
> > >
> > > On Tuesday, February 19, 2013 3:17:14 PM UTC+2, Zachary Tong wrote:
> > >
> > > > Elasticsearch will automatically elect a new master if the old one
> > > > becomes unavailable, but master's are not necessary for search/indexing
> > > > operations. E.g. any node in the cluster is fully capable of servicing a
> > > > search request, not just the master node.
> > > >
> > > > The master is in charge of managing cluster metadata and state
> > > > (mappings, new indices, etc), in addition to normal indexing/search
> > > > requirements.
> > > >
> > > > It is also important to have your minimum\_master\_nodes setting
> > > > configured properly, to help avoid split-brains.
> > > >
> > > > -Zach
> > > >
> > > > On Tuesday, February 19, 2013 7:59:53 AM UTC-5, Ophir Michaeli wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > What's the best practice for an elasticsearch client to handle a
> > > > > scenario in which elasticsearch master falls?
> > > > >
> > > > > From my tests I can connect to any of the slaves and continue on. Does
> > > > > it matter that the client connect to a slave and not to a master?
> > > > >
> > > > > Thanks,
> > > > > Ophir
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:50am UTC](https://discuss.elastic.co/t/master-fall/10799/7 "2017-07-06T02:50:34Z")