# Adding new node to cluster

**URL:** <https://discuss.elastic.co/t/adding-new-node-to-cluster/242223>\
**Category:** Elasticsearch\
**Created:** [July 22, 2020, 3:25pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223 "2020-07-22T15:25:07Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\
**Post date:** [July 22, 2020, 3:25pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/1 "2020-07-22T15:25:07Z")

</div>

Hi,  
I want to add new node to running cluster. I have two servers on which elastic search 7.8 is setup.

```
Server # 1
IP address x.x.x.x port 9200
hostname: es1.example.com

Server # 2
IP Address x.x.x.x port 9200
hostname: es2.example.com

```

As mentioned in document "Each address can be either an IP address or a hostname which resolves to one or more IP addresses via DNS." so I user hostname in configuration file

```
Server # 1 elasticsearch.yml
cluster.name: MySureFit-ES
node.name: ${HOSTNAME}
node.master: true
node.data: true
discovery.seed_hosts: ["es1.example.com", "es2.example.com"]

Server # 2 elasticsearch.yml
cluster.name: MySureFit-ES
node.name: ${HOSTNAME}
node.master: false
node.data: true
discovery.seed_hosts: ["es1.example.com", "es2.example.com"]

```

output of following request \_cluster/state/nodes is

```
{
  "cluster_name" : "MySureFit-ES",
  "cluster_uuid" : "TgTN0IDwRu6TMwAVsgVAyw",
  "nodes" : {
  "e41LhIkzT7uF_LUfO_otSA" : {
  "name" : "v4Dev-Sandbox",
  "ephemeral_id" : "aC7viR8UQcSEnNpJQio9Gw",
  "transport_address" : "127.0.0.1:9300",
  "attributes" : {
    "ml.machine_memory" : "33342156800",
    "xpack.installed" : "true",
    "transform.node" : "true",
    "ml.max_open_jobs" : "20"
   }
  }
 }
}

```

Can any one help me to resolve this issue.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 22, 2020, 9:20pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/2 "2020-07-22T21:20:52Z")

</div>

> [@hariskhalique](#):
>
> ```auto
> "transport_address" : "127.0.0.1:9300",
> 
> ```

You probably need to check out [`network.host`](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html).

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [July 23, 2020, 4:35am UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/3 "2020-07-23T04:35:37Z")

</div>

Agreed - check your existing nodes for that setting, likely it's 0.0.0.0 which is often simplest (bind to all interfaces) - the default is localhost which means other nodes can't talk to you.

Note if you are on Docker (as I see you are in a Dev Sandbox), you also usually set network.publish\_host - see [Docker Networking](https://www.elastic.co/blog/docker-networking)

---

<div class="post-metadata">

**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\
**Post date:** [July 23, 2020, 1:28pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/4 "2020-07-23T13:28:50Z")

</div>

Hi,  
I leave network.host to default which is 127.0.0.1:9200. I did not add transport\_address.

Should I set server # 1 transport\_address to 127.0.0.1:9200 and server # 2 to 127.0.0.1:9300 and both should have same.

I checked with internal IP but no luck. I set network.host to internal IP for both server.

---

<div class="post-metadata">

**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\
**Post date:** [July 23, 2020, 1:36pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/5 "2020-07-23T13:36:13Z")

</div>

> [@hariskhalique](#):
>
> `"transport_address" : "127.0.0.1:9300"`

I am not on docker I have two server purchase by aws. I do have private IPs with me when I set network.host to private IPs and set discovery.seed\_hosts to private IPs still having the same problem.

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [July 24, 2020, 1:20am UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/6 "2020-07-24T01:20:20Z")

</div>

I don't think you can get a cluster working with network.host set to 127.0.0.1 on your other 'working' nodes - set to 0.0.0.0 and they should listen on all IPs. You can also check this via:

netstat -ltnp that will show you what is listening on what - you'll see java listening on port 9200 and 9300, and either on \* or 127.0.0.1, etc

Don't think you should set transport\_address; I THINK it picks up network.host by default, but you can test.

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 26, 2020, 10:21pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/7 "2020-07-26T22:21:39Z")

</div>

`network.host` will apply to both REST and binary protocols (9200 and 9300).

---

<div class="post-metadata">

**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\
**Post date:** [July 27, 2020, 6:06pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/8 "2020-07-27T18:06:45Z")

</div>

Hi Guys,  
thanks for help after make changes, getting following error

```
[2020-07-27T14:49:52,424][WARN][o.e.c.c.ClusterFormationFailureHelper] [v4Dev-Sandbox] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [v4Dev-Sandbox, ES-New] to bootstrap a cluster: have discovered [{v4Dev-Sandbox}{wt5MqhDuQ2i0w7Gqyinugg}{AgzGQPtKQbq5Ni9Q7-F5yQ}{172.31.3.110}{172.31.3.110:9300}{dilmrt}{ml.machine_memory=33342156800, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}]; discovery will continue using [172.31.6.58:9300] from hosts providers and [{v4Dev-Sandbox}{wt5MqhDuQ2i0w7Gqyinugg}{AgzGQPtKQbq5Ni9Q7-F5yQ}{172.31.3.110}{172.31.3.110:9300}{dilmrt}{ml.machine_memory=33342156800, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0

```

Following is elasticsearch.yml

```
cluster.name: MySureFit-ES
node.name: ${HOSTNAME}
node.master: true
node.data: true
network.host: x.x.x.110
discovery.seed_hosts: ["x.x.x.110", "x.x.x.58"]

```

Output of the following curl -XGET '[http://x.x.x.110:9200](http://x.x.x.110:9200)'

```
{
   "name" : "v4Dev-Sandbox",
   "cluster_name" : "MySureFit-ES",
   "cluster_uuid" : "_na_",
   "version" : {
"number" : "7.8.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
"build_date" : "2020-06-14T19:35:50.234439Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}

```

Second server

```
cluster.name: MySureFit-ES
node.name: ${HOSTNAME}
node.master: false
node.data: true
network.host: x.x.x.58
discovery.seed_hosts: ["x.x.x.110", "x.x.x.58"]

```

Output of following command curl -XGET '[http://x.x.x.58:9200](http://x.x.x.58:9200)'

```
{
   "name" : "ES-New",
   "cluster_name" : "MySureFit-ES",
  "cluster_uuid" : "_na_",
  "version" : {
"number" : "7.8.0",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "757314695644ea9a1dc2fecd26d1a43856725e65",
"build_date" : "2020-06-14T19:35:50.234439Z",
"build_snapshot" : false,
"lucene_version" : "8.5.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
 },
 "tagline" : "You Know, for Search"
 }

```

Error from second server

```
 [2020-07-27T14:51:22,933][WARN][o.e.c.c.ClusterFormationFailureHelper] [ES-New] master not discovered yet: have discovered [{ES-New}{xvb57WoeTgml714sQsNvvg}{zYCMuLBtRa2pYNiNW2FwFQ}{172.31.6.58}{172.31.6.58:9300}{dilrt}{ml.machine_memory=8287748096, xpack.installed=true, transform.node=true, ml.max_open_jobs=20}, {v4Dev-Sandbox}{wt5MqhDuQ2i0w7Gqyinugg}{AgzGQPtKQbq5Ni9Q7-F5yQ}{172.31.3.110}{172.31.3.110:9300}{dilmrt}{ml.machine_memory=33342156800, ml.max_open_jobs=20, xpack.installed=true, transform.node=true}]; discovery will continue using [172.31.3.110:9300] from hosts providers and [] from last-known cluster state; node term 0, last-accepted version 0 in term 0

```

What I did wrong ?

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [July 28, 2020, 6:59am UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/9 "2020-07-28T06:59:47Z")

</div>

What is history here as your 'master' node v4Dev-Sandbox was never part of a cluster, but this whole thread is about having a working cluster and adding a node, so I'm confused.

But your early posting of v4Dev-Sandbox shows a cluster UUID, so it joined then, so how did we lose that? I assume we lost the self-bootstrapping mode with non-local IP, and might need three nodes to build cluster, but this error message is confusing.

@warkolm likely knows much more 😉

You've kinda changed server names from the original posting of es1, es2 to real names, etc. so I'm a bit lost.

---

<div class="post-metadata">

**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\
**Post date:** [July 28, 2020, 12:32pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/10 "2020-07-28T12:32:27Z")

</div>

According to you that it will not work with 127.0.0.1 so I change it to private IPs. I remove nodes data from /var/lib/elasticsearch/nodes.

When I restart again I am getting this error.

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [July 29, 2020, 6:35am UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/11 "2020-07-29T06:35:53Z")

</div>

But was v4Dev-Sandbox the master before? As it's starting now and not finding a cluster, or at least confused - can we get that node back up and running as the master of a one node cluster again so you have a stable starting point?

---

<div class="post-metadata">

**Author:** ![hariskhalique](https://avatars.discourse-cdn.com/v4/letter/h/a3d4f5/32.png) [@hariskhalique](https://discuss.elastic.co/u/hariskhalique)\
**Post date:** [July 29, 2020, 1:21pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/12 "2020-07-29T13:21:53Z")

</div>

> [@hariskhalique](#):
>
> `172.31.6.58`

When I remove

```auto
cluster.name: MySureFit-ES
node.name: ${HOSTNAME}
node.master: false
node.data: true
network.host: x.x.x.58
discovery.seed_hosts: ["x.x.x.110", "x.x.x.58"]

```

> node.master

from configuration every thing came back online and now its working and cluster have both nodes.

Thanks for the great help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 26, 2020, 1:32pm UTC](https://discuss.elastic.co/t/adding-new-node-to-cluster/242223/13 "2020-08-26T13:32:51Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.