# Recoverability from node replacement on a 2-node cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [January 22, 2026, 6:00am UTC](https://discuss.elastic.co/t/recoverability-from-node-replacement-on-a-2-node-cluster/384685 "2026-01-22T06:00:12Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![crazyzhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/crazyzhou/32/146716_2.png) [@crazyzhou](https://discuss.elastic.co/u/crazyzhou)\ **Post date:** [January 22, 2026, 6:00am UTC](https://discuss.elastic.co/t/recoverability-from-node-replacement-on-a-2-node-cluster/384685/1 "2026-01-22T06:00:12Z")
We are using eck-operator and running Elasticsearch cluster on Kubernetes. We have a 3 master-eligible nodes and several worker nodes in the cluster. In the case for running the cluster on 2 physical nodes, if a node goes down, we understand that Elasticsearch cluster is not resilient to failures. [Resilience in small clusters | Elastic Docs](https://www.elastic.co/docs/deploy-manage/production-guidance/availability-and-resilience/resilience-in-small-clusters#high-availability-cluster-design-two-nodes) Our question is when we replace with a fresh new node without the previous data, is there a technical way that we can still recover the cluster with the index data kept? Now we are hitting a split-brain issue in the master node leader election because we replaced the physical node which hosted 2 master-eligible node which the majority is lost, so we have to re-establish the cluster to recover. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [January 22, 2026, 7:15am UTC](https://discuss.elastic.co/t/recoverability-from-node-replacement-on-a-2-node-cluster/384685/2 "2026-01-22T07:15:53Z")
> [@crazyzhou](#): > > Our question is when we replace with a fresh new node without the previous data, is there a technical way that we can still recover the cluster with the index data kept? Now we are hitting a split-brain issue in the master node leader election because we replaced the physical node which hosted 2 master-eligible node which the majority is lost, so we have to re-establish the cluster to recover. There is no automatic way to recover the data if you permanently lose a majority of master eligible master nodes. Sometimes I believe it _ **may** _ be possible to reconfigure the cluster, but that is not guaranteed to succeed and requires the manual use of the [elasticsearch node utility](https://www.elastic.co/docs/reference/elasticsearch/command-line-tools/node-tool), which is hard (maybe even impossible) to use with k8s, and can result in data loss. In your scenario you will need to set up a new cluster and restore data from a recent snapshot taken using the official snapshot API to recover. ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [January 22, 2026, 3:01pm UTC](https://discuss.elastic.co/t/recoverability-from-node-replacement-on-a-2-node-cluster/384685/3 "2026-01-22T15:01:39Z")
Er, just don’t put your 3 master-eligible instances on only 2 physical nodes? Sorry, as you know the risks here, that’s just … unwise. The 3rd master-eligible instance’s node doesn’t need much in terms of resources. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [January 26, 2026, 10:32am UTC](https://discuss.elastic.co/t/recoverability-from-node-replacement-on-a-2-node-cluster/384685/4 "2026-01-26T10:32:57Z")
> [@Christian\_Dahlqvist](#): > > There is no automatic way to recover the data if you permanently lose a majority of master eligible master nodes. TBC this is not a limitation imposed by Elasticsearch, it is mathematically impossible for a 2-node cluster to be resilient to the loss of either node - see e.g. [this page](https://en.wikipedia.org/wiki/Two_Generals'_Problem). Please add a separate tiebreaker node as suggested in the [docs you linked](https://www.elastic.co/docs/deploy-manage/production-guidance/availability-and-resilience/resilience-in-small-clusters#high-availability-cluster-design-two-nodes-plus) rather than trying to find unsafe workarounds.