# Node addition failed

**URL:** <https://discuss.elastic.co/t/node-addition-failed/244523>\
**Category:** Elasticsearch\
**Created:** [August 11, 2020, 9:44am UTC](https://discuss.elastic.co/t/node-addition-failed/244523 "2020-08-11T09:44:27Z")\
**Posts on this page:** 15\
**Page:** 2

<div class="post-metadata">

**Author:** ![Faraz\_Ahmed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faraz_ahmed/32/73677_2.png) [@Faraz\_Ahmed](https://discuss.elastic.co/u/Faraz_Ahmed)\
**Post date:** [August 11, 2020, 2:56pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/21 "2020-08-11T14:56:09Z")

</div>

telnet working on port 9200 but not on 9300 on both nodes

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 11, 2020, 2:57pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/22 "2020-08-11T14:57:17Z")

</div>

It has to work on port 9300, so that would be the problem.

---

<div class="post-metadata">

**Author:** ![Faraz\_Ahmed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faraz_ahmed/32/73677_2.png) [@Faraz\_Ahmed](https://discuss.elastic.co/u/Faraz_Ahmed)\
**Post date:** [August 11, 2020, 2:58pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/23 "2020-08-11T14:58:46Z")

</div>

can you please guide me how to enable port 9300?

---

<div class="post-metadata">

**Author:** ![Faraz\_Ahmed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faraz_ahmed/32/73677_2.png) [@Faraz\_Ahmed](https://discuss.elastic.co/u/Faraz_Ahmed)\
**Post date:** [August 11, 2020, 3:03pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/24 "2020-08-11T15:03:48Z")

</div>

I manually set this parameter http.port: 9200 in config file. Can i change this to 9300?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [August 11, 2020, 3:05pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/25 "2020-08-11T15:05:59Z")

</div>

No. Elasticsearch uses port 9300 for internal communication by default so it is likely a firewall or network issue.

---

<div class="post-metadata">

**Author:** ![Faraz\_Ahmed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faraz_ahmed/32/73677_2.png) [@Faraz\_Ahmed](https://discuss.elastic.co/u/Faraz_Ahmed)\
**Post date:** [August 12, 2020, 6:40am UTC](https://discuss.elastic.co/t/node-addition-failed/244523/26 "2020-08-12T06:40:42Z")

</div>

I have disable firewall and checked network configuration but unable to solve the issue.

Is any tracing is available in elasticsearch to find the main root cause.

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [August 12, 2020, 8:38am UTC](https://discuss.elastic.co/t/node-addition-failed/244523/27 "2020-08-12T08:38:30Z")

</div>

What infrastructure is this on? AWS, local VMWare, Docker or what?

Suggest first making sure ES is listening on that port at OS level on BOTH hosts: "netstat -lntp" should show java listening on port 9200 and 9300, on the IP your list or on \* - if not, then you have a local elasticsearch.yml problem.

LIke this (0 0.0.0.0 is okay also):

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name  
tcp 0 0 :::9200 :::\* LISTEN 11936/java  
tcp 0 0 :::9300 :::\* LISTEN 11936/java

Make sure it's not just listening on 127.0.0.1 which should not be possible since you have "network.host: 192.168.3.181" (i.e. you don't want to see 127.0.0.1:9300) - also make sure that IP matches the host IP; we assume so since port 9200 works.

Check iptables on that host: 'iptables -vnL' ideally empty. Also check cloud network security groups which need to be open on port 9200 & 9300 to the LAN and other nodes.

In the end, both nodes must be able to telnet to the other on port 9300, the names must be right, and of course the IPs.

---

<div class="post-metadata">

**Author:** ![Faraz\_Ahmed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faraz_ahmed/32/73677_2.png) [@Faraz\_Ahmed](https://discuss.elastic.co/u/Faraz_Ahmed)\
**Post date:** [August 12, 2020, 9:06am UTC](https://discuss.elastic.co/t/node-addition-failed/244523/28 "2020-08-12T09:06:58Z")

</div>

Thank You so much for your reply.

\<[root@faraz ~]# netstat -lntp  
Active Internet connections (only servers)  
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name  
tcp 0 0 0.0.0.0:111 0.0.0.0:\* LISTEN 1/systemd  
tcp 0 0 192.168.122.1:53 0.0.0.0:\* LISTEN 1459/dnsmasq  
tcp 0 0 0.0.0.0:22 0.0.0.0:\* LISTEN 1042/sshd  
tcp 0 0 127.0.0.1:631 0.0.0.0:\* LISTEN 1043/cupsd  
tcp 0 0 127.0.0.1:25 0.0.0.0:\* LISTEN 1340/master  
tcp6 0 0 :::111 :::\* LISTEN 1/systemd  
tcp6 0 0 192.168.3.181:9200 :::\* LISTEN 3108/java  
tcp6 0 0 ::1:9300 :::\* LISTEN 3108/java  
tcp6 0 0 127.0.0.1:9300 :::\* LISTEN 3108/java  
tcp6 0 0 :::22 :::\* LISTEN 1042/sshd  
tcp6 0 0 ::1:631 :::\* LISTEN 1043/cupsd  
tcp6 0 0 ::1:25 :::\* LISTEN 1340/master

> 

iptables -vnL  
Chain INPUT (policy ACCEPT 359K packets, 38M bytes)  
pkts bytes target prot opt in out source destination  
0 0 ACCEPT udp -- virbr0 \* 0.0.0.0/0 0.0.0.0/0 udp dpt:53  
0 0 ACCEPT tcp -- virbr0 \* 0.0.0.0/0 0.0.0.0/0 tcp dpt:53  
0 0 ACCEPT udp -- virbr0 \* 0.0.0.0/0 0.0.0.0/0 udp dpt:67  
0 0 ACCEPT tcp -- virbr0 \* 0.0.0.0/0 0.0.0.0/0 tcp dpt:67

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)  
pkts bytes target prot opt in out source destination  
0 0 ACCEPT all -- \* virbr0 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED  
0 0 ACCEPT all -- virbr0 \* 192.168.122.0/24 0.0.0.0/0  
0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0  
0 0 REJECT all -- \* virbr0 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable  
0 0 REJECT all -- virbr0 \* 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT 33676 packets, 322M bytes)  
pkts bytes target prot opt in out source destination  
0 0 ACCEPT udp -- \* virbr0 0.0.0.0/0 0.0.0.0/0 udp dpt:68

---

<div class="post-metadata">

**Author:** ![Faraz\_Ahmed](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/faraz_ahmed/32/73677_2.png) [@Faraz\_Ahmed](https://discuss.elastic.co/u/Faraz_Ahmed)\
**Post date:** [August 12, 2020, 9:17am UTC](https://discuss.elastic.co/t/node-addition-failed/244523/29 "2020-08-12T09:17:47Z")

</div>

Sorry I forgot to mention this is local VMs on same base machine.

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [August 12, 2020, 11:15am UTC](https://discuss.elastic.co/t/node-addition-failed/244523/30 "2020-08-12T11:15:33Z")

</div>

Hmm, wonder how you got it on both \* and 127.0.0.1:9300 as I'd expect \*

And that's different from the 192.168.3.181:9200 - why is that listen different from the port 9300 listen? Might be something in there.

Try to telnet to both 127.0.0.1:9300 and 192.168.3.181:9300 to make sure both work (tricky in Linux as it may auto-loopback).

iptables hard to interpret on a local machine like that, but something networking must be wrong for the nodes to not reach each other on 9200 vs. 9300 - all I can suggest is you try both ports from both machines and keep digging.

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [August 12, 2020, 2:11pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/31 "2020-08-12T14:11:49Z")

</div>

> [@Steve\_Mushero](#):
>
> And that's different from the 192.168.3.181:9200 - why is that listen different from the port 9300 listen? Might be something in there.

Yeah, the config in the OP shows that it's only listening for transport connections on localhost:

> [@faraz](#):
>
> network.bind\_host: 192.168.3.237  
> network.publish\_host: 192.168.3.237  
> transport.host: localhost

That's almost certainly a mistake, you should delete that `transport.host` line.

---

<div class="post-metadata">

**Author:** ![faraz](https://avatars.discourse-cdn.com/v4/letter/f/278dde/32.png) [@faraz](https://discuss.elastic.co/u/faraz)\
**Post date:** [August 12, 2020, 4:09pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/32 "2020-08-12T16:09:37Z")

</div>

This command solved my all problems. Thank you so much for the support.

sudo sysctl -w vm.max\_map\_count=262144

---

<div class="post-metadata">

**Author:** ![faraz](https://avatars.discourse-cdn.com/v4/letter/f/278dde/32.png) [@faraz](https://discuss.elastic.co/u/faraz)\
**Post date:** [August 12, 2020, 4:14pm UTC](https://discuss.elastic.co/t/node-addition-failed/244523/33 "2020-08-12T16:14:02Z")

</div>

Yes I change it to my host IP and all is working fine.

---

<div class="post-metadata">

**Author:** ![Steve\_Mushero](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steve_mushero/32/22441_2.png) [@Steve\_Mushero](https://discuss.elastic.co/u/Steve_Mushero)\
**Post date:** [August 13, 2020, 3:03am UTC](https://discuss.elastic.co/t/node-addition-failed/244523/34 "2020-08-13T03:03:33Z")

</div>

Note you need to put that max\_map\_count change in your /etc/sysctl.conf file, too as sysctl changes are not persisted across reboots.

Best practice is:

1. Change the /etc/sysctl.conf file
2. Run 'sysctl -p' to load that file's contents; this is what is done at boot, making sure the changes are correct, get loaded, etc. (and load any other pending changes).
3. Run 'sysctl -a | grep max\_map\_count' to verify the changed value was loaded into the kernel.

Glad it's all working for you now.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 10, 2020, 3:03am UTC](https://discuss.elastic.co/t/node-addition-failed/244523/35 "2020-09-10T03:03:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/node-addition-failed/244523.md?page=1)