# Force a node to be the master **URL:** \ **Category:** Elasticsearch\ **Created:** [January 18, 2021, 7:40pm UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467 "2021-01-18T19:40:35Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Post date:** [January 18, 2021, 7:40pm UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/1 "2021-01-18T19:40:35Z")
Just curious, how to fix a broken cluster like this: Via official Helm release, I made a 3 nodes cluster, was working fine: - node-0 / node-1 / node-2 I scaled down the STS to 0, to remove all pods/nodes. Then when I **scale up, the first pod (node-0) cannot elect a master**, because the other pods (node-1/node-2) are not up (which is the normal behavior, to prevent split brain ES need a quorum, I am ok): ``` master not discovered or elected yet, an election requires at least 2 nodes with ids from [jyUoUGcASqO8kQMrjNWSlQ, W7CrWAy-SuOtiWPy9C7eHw, ujsS2swvTzGQcEv__DoNcQ], ``` **Without recreating the other node (lets say servers have burnt!), how can I "force" this alone node to become the master?** I tried `voting_config_exclusions` but this API requires ... an elected master. (`master_not_discovered_exception`). ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [January 18, 2021, 8:44pm UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/2 "2021-01-18T20:44:06Z")
> [@ebuildy](#): > > **Without recreating the other node (lets say servers have burnt!), how can I "force" this alone node to become the master?** If you only have one out of three nodes remaining then you have lost data: the cluster state is only stored on a majority of the master-eligible nodes, which might be the missing two. The solution is to form a new cluster and restore a recent snapshot. ---
**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Post date:** [January 18, 2021, 9:00pm UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/3 "2021-01-18T21:00:17Z")
Do'h! So in a real situation in production, if a split-brain occurs, we lost the cluster and data 😕 Why not an API "/\_cluster/state/master" to set master so we could try to recover without losing data? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [January 18, 2021, 9:05pm UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/4 "2021-01-18T21:05:26Z")
> [@ebuildy](#): > > So in a real situation in production, if a split-brain occurs, we lost the cluster and data 😕 No, because [in a real situation in production you have snapshots](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#important-settings-backups) from which to recover. Simultaneously losing two of your three masters should be extraordinarily rare. > [@ebuildy](#): > > Why not an API "/\_cluster/state/master" to set master so we could try to recover without losing data? You can't solve this with an API: you already lost data when the second master died. ---
**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Post date:** [January 18, 2021, 9:13pm UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/5 "2021-01-18T21:13:25Z")
well ok, thanks you. ---
**Author:** ![ebuildy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ebuildy/32/6070_2.png) [@ebuildy](https://discuss.elastic.co/u/ebuildy)\ **Post date:** [January 19, 2021, 9:12am UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/6 "2021-01-19T09:12:12Z")
I am benchmarking elasticsearch against vespa, I am a big advocate of ES and we [quote="DavidTurner, post:4, topic:261467"] No, because [in a real situation in production you have snapshots](https://www.elastic.co/guide/en/elasticsearch/reference/current/important-settings.html#important-settings-backups) from which to recover. Simultaneously losing two of your three masters should be extraordinarily rare. [/quote] yes, we have original data in HDFS (we are running a search engine, almost 1Po), but it takes some days to re-index everything, and data changes quite frequently (snapshoting is quite impossible for us). We have several cluster, with hot/cold architecture (docs can move from cold to hot cluster), up time is really crucial for us, that why I am looking for "disaster recover" solutions rather than immutable ones (also, this is more "comfortable" for our boss to say we can recover!). So the cluster state is distributed between master nodes and if we lose one master node we can lose data? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [January 19, 2021, 9:26am UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/7 "2021-01-19T09:26:23Z")
> [@ebuildy](#): > > So the cluster state is distributed between master nodes and if we lose one master node we can lose data? No, it's always stored on a majority of the masters. If you have three master-eligible nodes then that means two of them. Thus if you lose a single master-eligible node then it's fine, one of the other two will also have the latest state. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 16, 2021, 9:26am UTC](https://discuss.elastic.co/t/force-a-node-to-be-the-master/261467/8 "2021-02-16T09:26:26Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.