# Multi node cluster failing to connect

**URL:** <https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753>\
**Category:** Elasticsearch\
**Tags:** docker\
**Created:** [March 1, 2023, 10:54am UTC](https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753 "2023-03-01T10:54:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vanwoes](https://avatars.discourse-cdn.com/v4/letter/v/ba8739/32.png) [@vanwoes](https://discuss.elastic.co/u/vanwoes)\
**Post date:** [March 1, 2023, 10:54am UTC](https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753/1 "2023-03-01T10:54:47Z")

</div>

Hi,

I'm having an issue with a multi node elasticsearch cluster where the nodes are failing to join in a docker swarm.

```auto
received join request from [{es01}{SBn0YXX-RyuPcEsz3vgdjA}{0l0I2h0HRteijUgnwwmvqg}{es01}{10.0.0.69}{10.0.0.69:9300}{hmrs}{xpack.installed=true}] but could not connect back to the joining node

```

```auto
error.message":"[es01][10.0.0.69:9300] connect_timeout[30s]","error.stack_trace":"org.elasticsearch.transport.ConnectTransportException: [es01][10.0.0.69:9300] connect_timeout[30s]

```

I'm able to get it working if I don't export any of the ports on es01, but I need to export as external services need to be able to connect to this elasticsearch cluster.

I'm using a very similar docker compose file listed here [Install Elasticsearch with Docker | Elasticsearch Guide [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html) but using version 3.2 instead (not sure if that makes any difference?)

Are there any additional networking configurations that I need to add?

Thank you!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 1, 2023, 11:55pm UTC](https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753/2 "2023-03-01T23:55:19Z")

</div>

Welcome to our community! 😃

It'd help if you provided configs and full logs.

---

<div class="post-metadata">

**Author:** ![rpd](https://avatars.discourse-cdn.com/v4/letter/r/bc79bd/32.png) [@rpd](https://discuss.elastic.co/u/rpd)\
**Post date:** [March 16, 2023, 4:53am UTC](https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753/3 "2023-03-16T04:53:50Z")

</div>

Hello

I am also facing very similar issue.

My environment is as below: 3 master nodes, 3 ingest nodes, 18 data nodes

Docker Swarm on Debian 11 (Bullseye)  
Docker Version: 23.0.1  
Elasticsearch 7.17.6

Initially master nodes were not discovered although each container can curl the other on 9200 and 9300

After playing with discovery.seed\_hosts and network.publish\_host and transport.host, I was able to get the master election working.

As soon as the master is elected, I get below error on all master nodes:

\<es\_container\> | "stacktrace": ["io.netty.handler.ssl.SslHandshakeTimeoutException: handshake timed out after 10000ms",

Further following stacktrace keeps coming on all nodes:

\<es\_container\> | "stacktrace": ["org.elasticsearch.transport.ConnectTransportException: [\<es\_master\_node\>][10.2.252.86:9300] general node connection failure",

The general node connection failure is not specific to master nodes, it is occurring for all master, ingest and data nodes.

At the container level I have verified network connectivity.

Could this happen until the nodes recover their indices fully?

We have pretty heavy indices (about 150 GB each)

The same configuration was working earlier on docker version 20.x, but facing this issue with docker version 23.0.1 and es 7.17.6

Any help to troubleshoot this would be appreciated. How can I increase the debug output in elasticsearch logs ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [March 19, 2023, 11:23pm UTC](https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753/4 "2023-03-19T23:23:30Z")

</div>

Please start your own topic for this 🙂

---

<div class="post-metadata">

**Author:** ![rpd](https://avatars.discourse-cdn.com/v4/letter/r/bc79bd/32.png) [@rpd](https://discuss.elastic.co/u/rpd)\
**Post date:** [March 20, 2023, 4:29am UTC](https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753/5 "2023-03-20T04:29:02Z")

</div>

Yes thanks, apologies for hijacking the discussion! I did find a solution, so will post the same soon.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2023, 4:29am UTC](https://discuss.elastic.co/t/multi-node-cluster-failing-to-connect/326753/6 "2023-04-17T04:29:15Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
