2 node elasticsearch cluster for production like environment

[cool999 (cool)]

Hi Team,

From below document, I am trying to find if I can use 2 node cluster for production like environment. I know 3 node cluster is always better choice.

Resilience in small clusters | Elasticsearch Guide [8.11] | Elastic

link above says, if you create 2 node es cluster, then make one of them as node.master: false so that node will not participate in election when either node is unavailable, so we have one master (and may be data node also) and other as only data node (i.e not master role)

It also says, The cluster can tolerate the loss of the other master-ineligible node. So it means after having above setting, there is no impact if non-master node is down, not available for some time.

but its not clear what will happen if the master node goes down? (I know all master node activity like creation/deletion of indices or shard allocation will not happen) but will the other data node continue receiving writes and responding to client requests? If yes then what will be the impact until master node come back and join cluster?

below link says, if one fails the other node can handle its tasks.

We recommend you assign both nodes all other roles except master eligibility. If one node fails, the other node can handle its tasks

and here it says this configuration is not resilient to failure.

Because it’s not resilient to failures, we do not recommend deploying a two-node cluster in production.

The above two lines mismatching to me.

Below is another link from elastic which says, if single master node goes down writes will be blocked but what if other data node is still live, will it not accept and respond to requests meanwhile?

3 nodes in the cluster, 2 data and 1 master - why if one fails it takes the whole cluster down?

If you have a single master and it goes offline writes will no longer be allowed. If you make your two data nodes master eligible as well so you have 3master eligible nodes it should be able to handle one if the nodes going down without blocking writes.

I am trying to find the impact of single master node failure on 2 node cluster and reason to use three node cluster instead.

Thanks,

[Christian_Dahlqvist]

In a 2 node cluster all master eligible nodes need to be available for the cluster to function and accept writes.

[cool999 (cool)]

Hi @Christian_Dahlqvist, Thanks for your reply.

As I asked above, the link says, The cluster can tolerate the loss of the other master-ineligible node then what does this mean with what you said.

and will not the other data node be able to serve all requests if master node is down?

Thanks,

[leandrojmp (Leandro Pereira)]

A two node cluster is not tolerant to failures, if you lose the master, your entire cluster will be down until the master is back up running.

In the scenario where you have a master-eligbile node and a data-only node, if you lose the data-only node, then your cluster is still up and can answer to requests as the master-node is still up.

In the same scenario if you lose the master-node, then the data-only node will not be able to answer to requests as a master node is needed for the cluster to work, without a working master node your cluster will not work and answer to requests.

[cool999 (cool)]

@leandrojmp , Thank you