# Cluster with 2 nodes changing shard type

**URL:** <https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263>\
**Category:** Elasticsearch\
**Created:** [September 19, 2019, 4:20pm UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263 "2019-09-19T16:20:08Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [September 19, 2019, 4:20pm UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/1 "2019-09-19T16:20:09Z")

</div>

Hi,  
currently I have a cluster with just 2 nodes. every index is divided just in 1 shard (indices are very very small) and 1 replica. I know that a cluster with 2 nodes has several limits but this is the state of art.  
All the primaries shards are allocated on `node1`, so if `node1` is restarted I cannot write into those indices.  
is there a way to change this situation? I would like to avoid this interruption if node1 is restarted (manually by me or because there is a crash).  
Personally I was thinking that if a replica shard lives alone in the cluster automatically will be elected as primary in that moment.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 19, 2019, 4:41pm UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/2 "2019-09-19T16:41:18Z")

</div>

For a cluster to be highly available and accept writes with one node unavailable you need at least 3 master eligible nodes in the cluster.

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [September 19, 2019, 5:50pm UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/3 "2019-09-19T17:50:06Z")

</div>

So, with just two nodes I can have high availability for read operations but not for write? Is there a way to change the status of the replica shard if it remains alone?

If I rise the shard number from 1 to 2,is it possible that the write operation will choose the primary shard anyway?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 20, 2019, 5:06am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/5 "2019-09-20T05:06:47Z")

</div>

> [@rschirin](#):
>
> So, with just two nodes I can have high availability for read operations but not for write?

Correct. Reads should still work.

> [@rschirin](#):
>
> Is there a way to change the status of the replica shard if it remains alone?

I do not understand what you mean. Could you please explain?

> [@rschirin](#):
>
> If I rise the shard number from 1 to 2,is it possible that the write operation will choose the primary shard anyway?

You still need 3 nodes. With only 2 nodes no master can get elected with 1 node missing as a strict majority of nodes are needed (majority of 2 is 2). With 3 master eligible nodes the remaining 2 nodes can elect a master and writing can continue.

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [September 20, 2019, 6:27am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/6 "2019-09-20T06:27:10Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> You still need 3 nodes. With only 2 nodes no master can get elected with 1 node missing as a strict majority of nodes are needed (majority of 2 is 2). With 3 master eligible nodes the remaining 2 nodes can elect a master and writing can continue.

I set `discovery.zen.minimum_master_nodes: 1` so I suppose that if one node goes down, automatically the other one will elect itself `master` so in this way I was able also for write operations.  
for instance:

```
discovery.zen.ping.unicast.hosts: ["localhost", "node2"]
discovery.zen.minimum_master_nodes: 1

```

I'm still using Elastic 5.6 but I have also another cluster on 6.8 with this configuration

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 20, 2019, 6:45am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/7 "2019-09-20T06:45:04Z")

</div>

> [@rschirin](#):
>
> I set `discovery.zen.minimum_master_nodes: 1`

If you do this then Elasticsearch logs the following warning:

`[2019-09-20T07:40:41,956][WARN][o.e.d.z.ElectMasterService] [node-1] value for setting "discovery.zen.minimum_master_nodes" is too low. This can result in data loss! Please set it to at least a quorum of master-eligible nodes (current value: [1], total number of master-eligible nodes used for publishing in this round: [2])`

It's not kidding. [**This config will result in data loss**](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/modules-node.html#split-brain).

The requirement for three master-eligible nodes isn't a restriction coming from Elasticsearch. It's not even theoretically possible to build a highly available distributed system using only two nodes.

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [September 20, 2019, 6:49am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/8 "2019-09-20T06:49:54Z")

</div>

hi @DavidTurner, that is the message 😀

anyway, if I set that value and the `node1` is the only one available (`node2` is completely turned off) and my applications will try to contact the running one, I suppose that `node1` will be the master so will be available to write documents. isn't it?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [September 20, 2019, 7:08am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/9 "2019-09-20T07:08:27Z")

</div>

Setting it that was means both nodes could form their own one node clusters when they are both up, which will guaranteed lead to data loss.

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [September 20, 2019, 9:46am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/10 "2019-09-20T09:46:10Z")

</div>

> [@Christian\_Dahlqvist](#):
>
> Setting it that was means both nodes could form their own one node clusters when they are both up, which will guaranteed lead to data loss.

ok, but let me ask this: if the master `node1` goes down because I power it off and `node2` is elected master (because I set the `minimum_master:1`), all the `node2`'s replica shards then will be considered as primary or not?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 20, 2019, 10:16am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/11 "2019-09-20T10:16:21Z")

</div>

The behaviour is undefined if you have two master nodes and `discovery.zen.minimum_master_nodes: 1`. Anything could happen.

If you have three master nodes and `discovery.zen.minimum_master_nodes: 2` then yes when you switch one node off any lost primaries will be replaced by promoting replicas on the other two nodes.

---

<div class="post-metadata">

**Author:** ![rschirin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rschirin/32/45283_2.png) [@rschirin](https://discuss.elastic.co/u/rschirin)\
**Post date:** [September 20, 2019, 10:27am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/12 "2019-09-20T10:27:44Z")

</div>

Ok. Last thing: if I want to switch primary\_shard[0] from `node1` to `node2` (and replica\_shard[0] from `node2` to `node1`, is there a way to do that?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 20, 2019, 10:30am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/13 "2019-09-20T10:30:12Z")

</div>

No, there's no way to do that. Primaries and replicas are pretty much equivalent, so there's normally no reason to care where the primary is currently located.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 18, 2019, 10:30am UTC](https://discuss.elastic.co/t/cluster-with-2-nodes-changing-shard-type/200263/14 "2019-10-18T10:30:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
