# Node management and values

**URL:** <https://discuss.elastic.co/t/node-management-and-values/198793>\
**Category:** Elasticsearch\
**Created:** [September 9, 2019, 9:40pm UTC](https://discuss.elastic.co/t/node-management-and-values/198793 "2019-09-09T21:40:46Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Philip\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/philip_brown/32/49853_2.png) [@Philip\_Brown](https://discuss.elastic.co/u/Philip_Brown)\
**Post date:** [September 9, 2019, 9:40pm UTC](https://discuss.elastic.co/t/node-management-and-values/198793/1 "2019-09-09T21:40:46Z")

</div>

Howdy,  
I'm new to elasticsearch, and I've tried to look up FAQs and things but no luck so far, so.... here goes.

We have a 3+10 cluster I get to inherit administration of.  
3 master nodes, 10 data.  
I need to add some new, higher disk capacity nodes into the cluster. and then eventually cycle through all the old nodes, upgrading disks for each of them.  
its a production cluster, so.... yeah :-/

I notice that Every single node currently has an elasticsearch.yml file. They are practically identical across all nodes. Each define

```
gateway: 
  expected_nodes: 10

```

So.. when I want to add nodes, do I need to go change the file on every single node, and make sure it has the correct value?

secondly.. every single yml config file has

```
discovery: 
  zen: 
    minimum_master_nodes: 2
    ping: 
      unicast: 
        hosts: 
             - host01
             - host02
      ...

```

Since this is unicast, not multicast.. does this mean that, again, I have to go to every single host and change the file to identify new hosts that I want to add to the cluster?

I'm confused, because all the web search I do about "how to add a node to elasticsearch cluster" just say to create the config on the new node, start the new node, and it just AutoMagically happens. There's no mention of having to go edit files on any of the existing nodes.

but if so, what are the values in the `elasticsearch.yml` file for???

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 9, 2019, 9:55pm UTC](https://discuss.elastic.co/t/node-management-and-values/198793/2 "2019-09-09T21:55:35Z")

</div>

> [@Philip\_Brown](#):
>
> So.. when I want to add nodes, do I need to go change the file on every single node, and make sure it has the correct value?

You don't _have_ to, no. The `gateway.expected_nodes` setting is described [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html) and can safely be set to a number lower than the number of nodes in the cluster. It is optional, only has an effect on the master-eligible nodes, and only when completely restarting the cluster. I think I would leave it at `10` for now and then adjust it later if needed.

> [@Philip\_Brown](#):
>
> Since this is unicast, not multicast.. does this mean that, again, I have to go to every single host and change the file to identify new hosts that I want to add to the cluster?

Really you should only be listing the master-eligible nodes under `discovery.zen.ping.unicast.hosts`. That way you don't need to adjust it when you add a master-ineligible node like a data node.

---

<div class="post-metadata">

**Author:** ![Philip\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/philip_brown/32/49853_2.png) [@Philip\_Brown](https://discuss.elastic.co/u/Philip_Brown)\
**Post date:** [September 9, 2019, 11:18pm UTC](https://discuss.elastic.co/t/node-management-and-values/198793/3 "2019-09-09T23:18:17Z")

</div>

Hm...  
so.. in theory, I should be able to add the new node, listing just the masters.  
but .. it isnt working.  
and its giving me a bogus "no route to host" error, when tcpdump and telnet both says that traffic IS actually flowing from the new node to the current master node.

Log file entries:

NFO ][node] [oc2esdata11-oc2] starting ...  
[2019-09-09 16:11:38,835][INFO][transport] [oc2esdata11-oc2] publish\_address {1.2.3.4:9300}, bound\_addresses {1.2.3.4:9300}  
[2019-09-09 16:11:38,843][INFO][discovery] [oc2esdata11-oc2] elasticsearch\_logstash/eKYDs3CIRQOaniiJyLVGUg  
[2019-09-09 16:11:41,901][INFO][discovery.zen] [oc2esdata11-oc2] failed to send join request to master [{oc2esnode02-oc2}{SLDhR7foQwuCpwp5NsecUg}{6.7.8.9}{6.7.8.9:9300}{rack=vm, data=false, master=true}], reason [RemoteTransportException[[oc2esnode02-oc2][6.7.8.9:9300][internal:discovery/zen/join]]; nested: ConnectTransportException[[oc2esdata11-oc2][1.2.3.4:9300] connect\_timeout[30s]]; nested: NotSerializableExceptionWrapper[no\_route\_to\_host\_exception: No route to host]; ]

telnet test:

[root@oc2esdata11 oc2]# telnet 6.7.8.9 9300  
Trying 6.7.8.9...  
Connected to 6.7.8.9.  
Escape character is '^]'.

---

<div class="post-metadata">

**Author:** ![Philip\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/philip_brown/32/49853_2.png) [@Philip\_Brown](https://discuss.elastic.co/u/Philip_Brown)\
**Post date:** [September 9, 2019, 11:26pm UTC](https://discuss.elastic.co/t/node-management-and-values/198793/4 "2019-09-09T23:26:24Z")

</div>

PS: googled a "fix" for this, but new node does have this defined already:  
network:  
host: _enp4s0f0:ipv4_

and that IS the actual active network interface name.  
enp4s0f0: flags=4163\<UP,BROADCAST,RUNNING,MULTICAST\> mtu 1500

---

<div class="post-metadata">

**Author:** ![Philip\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/philip_brown/32/49853_2.png) [@Philip\_Brown](https://discuss.elastic.co/u/Philip_Brown)\
**Post date:** [September 10, 2019, 12:33am UTC](https://discuss.elastic.co/t/node-management-and-values/198793/5 "2019-09-10T00:33:27Z")

</div>

oh. I thought you only needed port 9300 to the master. but apparently you also need a connection back on 9300,even if NOT a master.  
So I guess I'll have to get the firewalling tweaked for that!

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [September 10, 2019, 5:09am UTC](https://discuss.elastic.co/t/node-management-and-values/198793/6 "2019-09-10T05:09:05Z")

</div>

Yes that's right, this exception is the master trying (and failing) to connect back to the new node, which means that the new node got through to the master ok.

---

<div class="post-metadata">

**Author:** ![Philip\_Brown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/philip_brown/32/49853_2.png) [@Philip\_Brown](https://discuss.elastic.co/u/Philip_Brown)\
**Post date:** [September 11, 2019, 8:08pm UTC](https://discuss.elastic.co/t/node-management-and-values/198793/7 "2019-09-11T20:08:25Z")

</div>

clearer error messages would have been nice :-/

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 9, 2019, 8:10pm UTC](https://discuss.elastic.co/t/node-management-and-values/198793/8 "2019-10-09T20:10:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
