Number of master nodes for a multi-node cluster

[jkhere (Jay)]

Hi,

I am planning to set up an elasticsearch cluster of around 160 nodes. As I have read in the documentation and some other forum posts, I need to set up dedicated master nodes and data nodes. The cluster will be Index heavy ( daily 5 million + documents updated) and 200,000 searches made against it.

I read in a post that the number of minimum master node required can be determined based on the below formula.

N <= 2 ? 1 : floor(N/2)+1 rule
( N - number of nodes in the cluster)

In my case, N = 160, so is minimum master nodes is 81 . Am i correct ? I was thinking of setting up on 3 Master nodes and rest of them as data nodes.

[Tek_Chand]

@jkhere,

@jkhere

I am planning to set up an elasticsearch cluster of around 160 nodes.

Are you want to use 160 servers in elasticsaerch cluster or 160 are the servers which will send the logs to ELK?

@jkhere

(N/2)+1 rule

Yes..this is the rule to set the minimum master node in a cluster. Your node should be in odd number like 3,5 etc.

Please refer the below link it may help you:

https://qbox.io/blog/split-brain-problem-elasticsearch

Thanks.

[DavidTurner]

@jkhere

I read in a post that the number of minimum master node required can be determined based on the below formula.

I think you're misunderstanding what you've read. This formula is for the discovery.zen.minimum_master_nodes setting, where N is the total number of master-eligible nodes in your cluster. N should be much smaller than 160, and normally 3 is the right number.

Furthermore this setting is deprecated and ignored in Elasticsearch 7 so this formula is not at all relevant any more.

@jkhere

I was thinking of setting up on 3 Master nodes and rest of them as data nodes.

Yes, do that.

[jkhere (Jay)]

@DavidTurner

Furthermore this setting is deprecated and ignored in Elasticsearch 7 so this formula is not at all relevant any more.

Thank you for the clarification. I just read the elastic blog post on verison 7 announcement. I understand that it has changed.

We don't have any plans for setting up machine learning jobs or data ingest pipeline. So those will be set as false. However we do have a large volume of searches and very few heavy searches. It does not involve any large aggregations or analytics. So do i need any dedicated coordinating nodes ? I believe data nodes will also be acting as coordinating nodes.

[Tek_Chand]

@DavidTurner

I think you're misunderstanding what you've read. This formula is for the discovery.zen.minimum_master_nodes setting, where N is the total number of master-eligible nodes in your cluster.

Ah..sorry..Thanks for correcting me..i came back here to correct myself but you already posted. Somehow i wrote that. Yes, (N/2)+1 rule is used to set discovery.zen.minimum_master_nodes to avoid split brain problem and master node should be in odd number.

Thanks.