Hi,
What is the guideline on recommended number of master eligible nodes in a
cluster?
We have a big cluster with 90+ nodes, currently all are eligible to be
masters. However I do see master election take very long after a cluster
restart - most nodes are still trying to ping the old master and I think
there is no way to force a re-election.
Will re-election be faster if number of master eligible nodes be less? Is
there a rule of thumb, given N nodes how many should we configure to be
eligible to be masters?
The formula is N/2+1, so in your case that would be 46 if there are 90
nodes.
/JZ
On Wednesday, January 14, 2015 at 9:38:01 AM UTC+1, Darshat Shah wrote:
Hi,
What is the guideline on recommended number of master eligible nodes in a
cluster?
We have a big cluster with 90+ nodes, currently all are eligible to be
masters. However I do see master election take very long after a cluster
restart - most nodes are still trying to ping the old master and I think
there is no way to force a re-election.
Will re-election be faster if number of master eligible nodes be less? Is
there a rule of thumb, given N nodes how many should we configure to be
eligible to be masters?
That's the minimum number. Is it recommended to have more, or will
elections take longer if we have more than this minimum?
On Wednesday, January 14, 2015 at 2:16:27 PM UTC+5:30, drjz wrote:
The formula is N/2+1, so in your case that would be 46 if there are 90
nodes.
/JZ
On Wednesday, January 14, 2015 at 9:38:01 AM UTC+1, Darshat Shah wrote:
Hi,
What is the guideline on recommended number of master eligible nodes in a
cluster?
We have a big cluster with 90+ nodes, currently all are eligible to be
masters. However I do see master election take very long after a cluster
restart - most nodes are still trying to ping the old master and I think
there is no way to force a re-election.
Will re-election be faster if number of master eligible nodes be less? Is
there a rule of thumb, given N nodes how many should we configure to be
eligible to be masters?
To clarify, the guideline I'm looking for is how many nodes to set apart as
dedicated to be master eligible in a cluster of size N. That is not N/2+1.
On Wednesday, January 14, 2015 at 2:24:29 PM UTC+5:30, Darshat Shah wrote:
That's the minimum number. Is it recommended to have more, or will
elections take longer if we have more than this minimum?
On Wednesday, January 14, 2015 at 2:16:27 PM UTC+5:30, drjz wrote:
The formula is N/2+1, so in your case that would be 46 if there are 90
nodes.
/JZ
On Wednesday, January 14, 2015 at 9:38:01 AM UTC+1, Darshat Shah wrote:
Hi,
What is the guideline on recommended number of master eligible nodes in
a cluster?
We have a big cluster with 90+ nodes, currently all are eligible to be
masters. However I do see master election take very long after a cluster
restart - most nodes are still trying to ping the old master and I think
there is no way to force a re-election.
Will re-election be faster if number of master eligible nodes be less?
Is there a rule of thumb, given N nodes how many should we configure to be
eligible to be masters?
For installations like yours we would recommend 3 master eligible nodes.
This number doesn't need to scale with data volumes as your data nodes do.
A master eligible node should also be dedicated to that task alone - trying
to also fulfil search or indexing activities puts them at the risk of
becoming unresponsive.
We recommend 3 master-eligible nodes because that gives you both a
resiliency to failure and a necessary quorum required to avoid split-brain.
Fortunately master nodes can be lighter-weight machines than those
dedicated to search.
Anything less than 3 dedicated master-eligible nodes is essentially an
architectural compromise made for smaller-budget configurations.
On Wednesday, January 14, 2015 at 9:39:47 AM UTC, Darshat Shah wrote:
To clarify, the guideline I'm looking for is how many nodes to set apart
as dedicated to be master eligible in a cluster of size N. That is not
N/2+1.
On Wednesday, January 14, 2015 at 2:24:29 PM UTC+5:30, Darshat Shah wrote:
That's the minimum number. Is it recommended to have more, or will
elections take longer if we have more than this minimum?
On Wednesday, January 14, 2015 at 2:16:27 PM UTC+5:30, drjz wrote:
The formula is N/2+1, so in your case that would be 46 if there are 90
nodes.
/JZ
On Wednesday, January 14, 2015 at 9:38:01 AM UTC+1, Darshat Shah wrote:
Hi,
What is the guideline on recommended number of master eligible nodes in
a cluster?
We have a big cluster with 90+ nodes, currently all are eligible to be
masters. However I do see master election take very long after a cluster
restart - most nodes are still trying to ping the old master and I think
there is no way to force a re-election.
Will re-election be faster if number of master eligible nodes be less?
Is there a rule of thumb, given N nodes how many should we configure to be
eligible to be masters?
3 should be fine, and they don't even have to be beefy machines.
On Wednesday, January 14, 2015 at 1:39:47 AM UTC-8, Darshat Shah wrote:
To clarify, the guideline I'm looking for is how many nodes to set apart
as dedicated to be master eligible in a cluster of size N. That is not
N/2+1.
On Wednesday, January 14, 2015 at 2:24:29 PM UTC+5:30, Darshat Shah wrote:
That's the minimum number. Is it recommended to have more, or will
elections take longer if we have more than this minimum?
On Wednesday, January 14, 2015 at 2:16:27 PM UTC+5:30, drjz wrote:
The formula is N/2+1, so in your case that would be 46 if there are 90
nodes.
/JZ
On Wednesday, January 14, 2015 at 9:38:01 AM UTC+1, Darshat Shah wrote:
Hi,
What is the guideline on recommended number of master eligible nodes in
a cluster?
We have a big cluster with 90+ nodes, currently all are eligible to be
masters. However I do see master election take very long after a cluster
restart - most nodes are still trying to ping the old master and I think
there is no way to force a re-election.
Will re-election be faster if number of master eligible nodes be less?
Is there a rule of thumb, given N nodes how many should we configure to be
eligible to be masters?
I´ve created a 200 nodes cluster and successed with 3 master_eligible
nodes. The important thing is that they have the data_node setting in
false, and data_nodes have it the other way around. This will improve a lot
your cluster resiliency.
On Wednesday, January 14, 2015 at 4:22:58 PM UTC-3, Ed Kim wrote:
3 should be fine, and they don't even have to be beefy machines.
On Wednesday, January 14, 2015 at 1:39:47 AM UTC-8, Darshat Shah wrote:
To clarify, the guideline I'm looking for is how many nodes to set apart
as dedicated to be master eligible in a cluster of size N. That is not
N/2+1.
On Wednesday, January 14, 2015 at 2:24:29 PM UTC+5:30, Darshat Shah wrote:
That's the minimum number. Is it recommended to have more, or will
elections take longer if we have more than this minimum?
On Wednesday, January 14, 2015 at 2:16:27 PM UTC+5:30, drjz wrote:
The formula is N/2+1, so in your case that would be 46 if there are 90
nodes.
/JZ
On Wednesday, January 14, 2015 at 9:38:01 AM UTC+1, Darshat Shah wrote:
Hi,
What is the guideline on recommended number of master eligible nodes
in a cluster?
We have a big cluster with 90+ nodes, currently all are eligible to be
masters. However I do see master election take very long after a cluster
restart - most nodes are still trying to ping the old master and I think
there is no way to force a re-election.
Will re-election be faster if number of master eligible nodes be less?
Is there a rule of thumb, given N nodes how many should we configure to be
eligible to be masters?
@Matias_Waisgold : With your cluster of 200 nodes with 3 master_eligble, does all nodes vote for the master election or only masters ? If only masters vote, what is the configuration of above parameters ?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.