# Node will not join cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [November 4, 2019, 5:53pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456 "2019-11-04T17:53:57Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Terran](https://avatars.discourse-cdn.com/v4/letter/t/94ad74/32.png) [@Terran](https://discuss.elastic.co/u/Terran)\ **Post date:** [November 4, 2019, 5:53pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/1 "2019-11-04T17:53:57Z")
Running version 7.3.1. I have a single node cluster and this is operating fine. I want to join two other nodes to this cluster. Each time I try I either get another single node cluster, or elasticsearch will not start. Here is the output of the yml on the single node. # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: Elasticsearch\_Test # ------------------------------------ Node ------------------------------------ # Use a descriptive name for the node: node.name: master\_node # ---------------------------------- Network ----------------------------------- network.host: a.b.c.d # Set a custom port for HTTP: http.port: 9200 # --------------------------------- Discovery ---------------------------------- discovery.seed\_hosts: ["master\_node", "new\_node1", " new\_node2"] cluster.initial\_master\_nodes: master\_node # ---------------------------------- Gateway ----------------------------------- # Block initial recovery after a full cluster restart until N nodes are started: #gateway.recover\_after\_nodes: 3 Output of the new node.yml file # ---------------------------------- Cluster ----------------------------------- cluster.name: Elasticsearch\_Test # ------------------------------------ Node ------------------------------------ node.name: new\_node1 # ---------------------------------- Network ----------------------------------- network.host: a.b.c.d # Set a custom port for HTTP: http.port: 9200 # --------------------------------- Discovery ---------------------------------- discovery.seed\_providers: master\_node discovery.seed\_hosts: ["new\_node1", " new\_node2", "master\_node"] cluster.initial\_master\_nodes: master\_node minimum\_master\_nodes: 2 # ---------------------------------- Gateway ----------------------------------- gateway.recover\_after\_nodes: 2 Trying to start elasticsearch on new\_node1 with these files causes ES to error out and not start. I can't figure out if I have a configuration wrong or if something is preventing these hosts from connecting. ---
**Author:** ![Mohammad.ali](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammad.ali/32/45665_2.png) [@Mohammad.ali](https://discuss.elastic.co/u/Mohammad.ali)\ **Post date:** [November 4, 2019, 6:17pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/2 "2019-11-04T18:17:02Z")
Hello @Terran it would be great if you send the error log ---
**Author:** ![Terran](https://avatars.discourse-cdn.com/v4/letter/t/94ad74/32.png) [@Terran](https://discuss.elastic.co/u/Terran)\ **Post date:** [November 4, 2019, 7:18pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/3 "2019-11-04T19:18:43Z")
After doing some work with the logs and cleaning up some bad entries on the .yml file I am left with this as an output on my logs. [root@new\_node1 elasticsearch]# curl -XGET "http://new\_node1.tcore.com:9200/\_cluster/health" {"error":{"root\_cause":[{"type":"master\_not\_discovered\_exception","reason":null}],"type":"master\_not\_discovered\_exception","reason":null},"status":503}[root@new\_node1 elasticsearch]# [2019-11-04T11:11:35,282][INFO][o.e.c.c.JoinHelper] [new\_node1] failed to join {master\_node}{CDuQWuHvTbCNQgjRw6TPFw}{LVW2CAYnQK23hC2\_fQ53UQ}{10.6.48.235}{10.6.48.235:9300}{dilm}{ml.machine\_memory=8203476992, ml.max\_open\_jobs=20, xpack.installed=true} with JoinRequest{sourceNode={new\_node1}{5ptJjwi-S2-t6X4yv8fufg}{rToluebFR5eeXHYTdfxuUA}{10.6.48.233}{10.6.48.233:9300}{dilm}{ml.machine\_memory=8203476992, xpack.installed=true, ml.max\_open\_jobs=20}, optionalJoin=Optional[Join{term=23, lastAcceptedTerm=0, lastAcceptedVersion=0, sourceNode={new\_node1}{5ptJjwi-S2-t6X4yv8fufg}{rToluebFR5eeXHYTdfxuUA}{10.6.48.233}{10.6.48.233:9300}{dilm}{ml.machine\_memory=8203476992, xpack.installed=true, ml.max\_open\_jobs=20}, targetNode={master\_node}{CDuQWuHvTbCNQgjRw6TPFw}{LVW2CAYnQK23hC2\_fQ53UQ}{10.6.48.235}{10.6.48.235:9300}{dilm}{ml.machine\_memory=8203476992, ml.max\_open\_jobs=20, xpack.installed=true}}]} org.elasticsearch.transport.RemoteTransportException: [master\_node][10.6.48.235:9300][internal:cluster/coordination/join] Caused by: org.elasticsearch.transport.ConnectTransportException: [new\_node1][10.6.48.233:9300] connect\_exception Caused by: java.io.IOException: **No route to host: 10.6.48.233/10.6.48.233:9300** Caused by: java.io.IOException: No route to host .233 is the new node and .235 is the master. From what is looks like though it seems that .233 (new\_node1) can't talk to itself? Am I reading that right? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 4, 2019, 9:01pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/4 "2019-11-04T21:01:49Z")
> [@Terran](#): > > .233 is the new node and .235 is the master. From what is looks like though it seems that .233 (new\_node1) can't talk to itself? Am I reading that right? Not quite, this exception indicates that the node can talk to the master but the master can't connect back to the new node. Elasticsearch forms connections in both directions. ---
**Author:** ![Terran](https://avatars.discourse-cdn.com/v4/letter/t/94ad74/32.png) [@Terran](https://discuss.elastic.co/u/Terran)\ **Post date:** [November 5, 2019, 5:19pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/5 "2019-11-05T17:19:15Z")
I have checked my network status and I don't see anything that would prevent connections. I checked the port settings and I see this: tcp 0 0 0.0.0.0:22 0.0.0.0:\* LISTEN 918/sshd tcp 0 0 127.0.0.1:25 0.0.0.0:\* LISTEN 1414/master tcp6 0 0 10.6.48.233:9200 :::\* LISTEN 6540/java tcp6 0 0 10.6.48.233:9300 :::\* LISTEN 6540/java tcp6 0 0 :::22 :::\* LISTEN 918/sshd tcp6 0 0 ::1:25 :::\* LISTEN 1414/master It looks like 9300 is open so I am scratching my head. ---
**Author:** ![Terran](https://avatars.discourse-cdn.com/v4/letter/t/94ad74/32.png) [@Terran](https://discuss.elastic.co/u/Terran)\ **Post date:** [November 5, 2019, 6:22pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/6 "2019-11-05T18:22:28Z")
I was showing my troubles to a teammate and he pointed out that 9200 and 9300 both show binds to tcp6. This doesn't seem right, but I am not sure how I change that. I have attempted to disable IPv6 across the entire server, but these won't switch to IPv4. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 6, 2019, 10:02am UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/7 "2019-11-06T10:02:55Z")
I don't think `tcp6` is anything to worry about - this is normal and means that it's using the `AF_INET6` address family which [supports IPv4 and IPv6](https://unix.stackexchange.com/questions/237731/why-are-ipv4-tcp-connections-showing-as-tcp6/237747). Does `curl http://10.6.48.233:9300/` return `This is not an HTTP port` on the master node? ---
**Author:** ![Terran](https://avatars.discourse-cdn.com/v4/letter/t/94ad74/32.png) [@Terran](https://discuss.elastic.co/u/Terran)\ **Post date:** [November 6, 2019, 1:26pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/8 "2019-11-06T13:26:41Z")
I was not quite sure which server you wanted me to run this on, so I did both. the results for the curl command from the new node (.223) to itself is: This is not an HTTP port[root@new\_node1 The results from the master node (.235) to the new node (.233) is: curl: (7) Failed connect to 10.6.48.233:9300; Connection timed out ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 6, 2019, 3:20pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/9 "2019-11-06T15:20:16Z")
Ok, that tells us that `tcp6` isn't a problem since the call worked on the data node, but that there is something in your network configuration preventing the master node connecting to the data node. ---
**Author:** ![Terran](https://avatars.discourse-cdn.com/v4/letter/t/94ad74/32.png) [@Terran](https://discuss.elastic.co/u/Terran)\ **Post date:** [November 6, 2019, 4:32pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/10 "2019-11-06T16:32:04Z")
I do not have any firewall or the like between these hosts and they all share a network. (.233, .234, .235). Is there some sort of port settings for the server I need to configure? the below shows outputs on the correct ports, but am I receiving correctly? ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/c/2c98ec02ae6b2d2e445df86ba79bdab07b6d63ec.png) ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [November 6, 2019, 8:57pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/11 "2019-11-06T20:57:39Z")
The screenshot you've shared looks normal to me, but there are many other things that could be causing this connectivity issue. I can't really help much more with this kind of issue since it depends so much on your environment. ---
**Author:** ![Terran](https://avatars.discourse-cdn.com/v4/letter/t/94ad74/32.png) [@Terran](https://discuss.elastic.co/u/Terran)\ **Post date:** [November 11, 2019, 12:20pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/12 "2019-11-11T12:20:20Z")
I have resolved the issue. Turns out you should disable firewalld... Thanks for the help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 9, 2019, 12:20pm UTC](https://discuss.elastic.co/t/node-will-not-join-cluster/206456/13 "2019-12-09T12:20:25Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.