# Elastic node reads previous cluster uuid from data folder even if it is re-configured to join another cluster

**URL:** <https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471>\
**Category:** Elasticsearch\
**Created:** [May 3, 2019, 5:44am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471 "2019-05-03T05:44:43Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![shreymath9999](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreymath9999/32/45471_2.png) [@shreymath9999](https://discuss.elastic.co/u/shreymath9999)\
**Post date:** [May 3, 2019, 5:44am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/1 "2019-05-03T05:44:43Z")

</div>

Here's the case,

**In brief** :  
On bootstrapping a single node, it forms a single node cluster and stores the cluster uuid in its data folder. Now even if it is re-configured to join a multi-node cluster in its yaml file. It will still refer that old cluster uuid from data folder and will not join the new cluster.

**In detail** :  
Create a single node (say _node-x_) with below configuration and set it up,

```
cluster.name: elasticsearch_test
node.name: node-x
path.data: C:\ProgramData\Elastic\Elasticsearch\data
path.logs: C:\ProgramData\Elastic\Elasticsearch\logs
http.port: 9200
network.host: 127.0.0.1
transport.tcp.port: 9300

```

Now set up 2 more nodes (_node-2_ and _node-3_) with below configurations to form a cluster,

```
cluster.name: elasticsearch_test
node.name: node-2
path.data: C:\ProgramData\Elastic\Elasticsearch_node_2\data
path.logs: C:\ProgramData\Elastic\Elasticsearch_node_2\logs
http.port: 9201
network.host: 127.0.0.1
discovery.seed_hosts: ["127.0.0.1:9302","127.0.0.1:9300","127.0.0.1:9301"]
cluster.initial_master_nodes: ["node-x", "node-2", "node-3"]
transport.tcp.port: 9301

```

You will observe, _node-x_ is not included in the cluster **elasticsearch\_test** yet,  
[http://localhost:9201/\_cat/nodes](http://localhost:9201/_cat/nodes)

Now update the configuration on _node-x_ with below properties and restart _node-x_ to make it join the **elasticsearch\_test** cluster.

```
discovery.seed_hosts: ["127.0.0.1:9302","127.0.0.1:9300","127.0.0.1:9301"]
cluster.initial_master_nodes: ["node-x", "node-2", "node-3"]

```

You will observe, _node-x_ has form its own cluster referring the uuid of its last cluster configuration from data folder and haven't joined cluster **elasticsearch\_test**.

Solution : Now delete the contents of data folder of _node-x_ and restart _node-x_. It will join the cluster **elasticsearch\_test**.

As per Elasticsearch document 7.x :  
[[Bootstrapping a cluster | Elasticsearch Guide [7.16] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/modules-discovery-bootstrap-cluster.html)]([http://Bootstrapping](http://Bootstrapping) a cluster 7.x)

> You must set `cluster.initial_master_nodes` to the same list of nodes on each node on which it is set in order to be sure that only a single cluster forms during bootstrapping and therefore to avoid the risk of data loss.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 3, 2019, 6:21am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/2 "2019-05-03T06:21:05Z")

</div>

I think you can use this tool: [https://www.elastic.co/guide/en/elasticsearch/reference/7.x/node-tool.html#node-tool-detach-cluster](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/node-tool.html#node-tool-detach-cluster)

Never tried it yet though 😉

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 3, 2019, 6:52am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/3 "2019-05-03T06:52:15Z")

</div>

This is not the right time to use `elasticsearch-node`. It's only appropriate to use it like this when you have experienced a disaster and have no other options, such as restoring from a snapshot.

The process described above is broken. It might have seemed to have worked in earlier versions, but there was a risk of data loss when doing what you describe. You are forming multiple clusters, and Elasticsearch is now rightly refusing to merge them together later. The solution is quoted above:

> You must set `cluster.initial_master_nodes` to the same list of nodes on each node on which it is set in order to be sure that only a single cluster forms during bootstrapping and therefore to avoid the risk of data loss.

You are not setting `cluster.initial_master_nodes` to the same list of nodes on each node, because the first time you start up `node-x` it is not set.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 3, 2019, 7:03am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/4 "2019-05-03T07:03:27Z")

</div>

> [@DavidTurner](#):
>
> You are not setting `cluster.initial_master_nodes` to the same list of nodes on each node, because the first time you start up `node-x` it is not set.

Sorry, it's actually a little more subtle than that. You are starting this first node up in development mode, which [causes it to set `cluster.initial_master_nodes` itself](https://www.elastic.co/guide/en/elasticsearch/reference/7.x/modules-discovery-bootstrap-cluster.html#_auto_bootstrapping_in_development_mode). However the fix is still the same: if you want to be sure to form a single cluster, you should set `cluster.initial_master_nodes` explicitly on every master-eligible node until the cluster has formed.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [May 3, 2019, 7:58am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/5 "2019-05-03T07:58:47Z")

</div>

Many thanks @DavidTurner for fixing my lack of knowledge on this. 🤗

---

<div class="post-metadata">

**Author:** ![shreymath9999](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shreymath9999/32/45471_2.png) [@shreymath9999](https://discuss.elastic.co/u/shreymath9999)\
**Post date:** [May 3, 2019, 10:11am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/6 "2019-05-03T10:11:03Z")

</div>

Because first time I want to make _node-x_ join any other cluster than why would I set

> cluster.initial\_master\_nodes

and my question here is very simple. Why the _node-x_ is referring cluster details from data folder rather than reading the configuration file again ?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 3, 2019, 10:20am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/7 "2019-05-03T10:20:03Z")

</div>

> [@shreymath9999](#):
>
> Why the _node-x_ is referring cluster details from data folder rather than reading the configuration file again ?

It would be a very bad idea to read the config file again, because this would let you perform the unsafe sequence of steps you describe in your original post and potentially lose data as a result.

`cluster.initial_master_nodes` is only needed the first time you start up a cluster. After that, it is ignored. You might be interested in the section entitled **Safety First** in [this blog post](https://www.elastic.co/blog/a-new-era-for-cluster-coordination-in-elasticsearch) for more information.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 31, 2019, 10:20am UTC](https://discuss.elastic.co/t/elastic-node-reads-previous-cluster-uuid-from-data-folder-even-if-it-is-re-configured-to-join-another-cluster/179471/8 "2019-05-31T10:20:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
