# High availability of system indices: how-to mitigate system index failures

**URL:** https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082
**Category:** Elasticsearch
**Created:** [February 22, 2021, 2:50pm UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082 "2021-02-22T14:50:31Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![bertr](https://avatars.discourse-cdn.com/v4/letter/b/e495f1/32.png) [@bertr](https://discuss.elastic.co/u/bertr)
#### Post date: [February 22, 2021, 2:50pm UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082/1 "2021-02-22T14:50:31Z")

</div>

I notice that system indices in Elasticsearch have "index.number\_of\_replicas": "0" and this cannot be updated. Is there a reason why this cannot be set to 1 (or more) and how safe is it to have no replicas for system indices from a high availability point of view? How can we mitigate system index failures?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [February 22, 2021, 11:29pm UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082/2 "2021-02-22T23:29:15Z")

</div>

They should be set as [auto-expanding](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html) using the `index.auto_expand_replicas` setting in the relevant template.

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [February 23, 2021, 4:58am UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082/3 "2021-02-23T04:58:45Z")

</div>

How many data nodes do you have in your cluster?

---

<div class="post-metadata">

### Author: ![bertr](https://avatars.discourse-cdn.com/v4/letter/b/e495f1/32.png) [@bertr](https://discuss.elastic.co/u/bertr)
#### Post date: [February 23, 2021, 8:31am UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082/4 "2021-02-23T08:31:22Z")

</div>

We are investigating two types of clusters: 2 clusters with 3 nodes and 1 having 6 nodes. The 2 clusters having 3 nodes are in separate data centers and will be used as multi cluster HA implementation (Active-Passive).  
The single 6 node cluster has 3 nodes in each datacenter and should act as Active-Active HA solution.  
I know how to deal with zones for replicas, cluster replication etc, but I am not sure how-to HA the system indices in each solution.  
Mayve allsystem indeices are secretly available on all (data) nodes? That would be nice!

---

<div class="post-metadata">

### Author: ![bertr](https://avatars.discourse-cdn.com/v4/letter/b/e495f1/32.png) [@bertr](https://discuss.elastic.co/u/bertr)
#### Post date: [February 23, 2021, 9:32am UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082/5 "2021-02-23T09:32:14Z")

</div>

OK, I close this topic. Apparently when you start with multi node clusters, all system indices do have replicas. The settings show the following attributes:  
"index.auto\_expand\_replicas": "0-1",  
"index.number\_of\_replicas": "1"

It seems that system indices high availability is taken care of automatically. Great!

---

<div class="post-metadata">

### Author: ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)
#### Post date: [February 23, 2021, 10:23am UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082/6 "2021-02-23T10:23:10Z")

</div>

> [@bertr](#):
>
> The single 6 node cluster has 3 nodes in each datacenter and should act as Active-Active HA solution.

Note that ([as per the docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/high-availability-cluster-design-large-clusters.html#high-availability-cluster-design-two-zones)) this doesn't work as a HA solution:

> You cannot configure a two-zone cluster so that it can tolerate the loss of _either_ zone because this is theoretically impossible.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 23, 2021, 10:23am UTC](https://discuss.elastic.co/t/high-availability-of-system-indices-how-to-mitigate-system-index-failures/265082/7 "2021-03-23T10:23:31Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
