# Failed to connect to master node

**URL:** <https://discuss.elastic.co/t/failed-to-connect-to-master-node/119754>\
**Category:** Elasticsearch\
**Created:** [February 14, 2018, 9:07am UTC](https://discuss.elastic.co/t/failed-to-connect-to-master-node/119754 "2018-02-14T09:07:22Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sbu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbu/32/27718_2.png) [@sbu](https://discuss.elastic.co/u/sbu)\
**Post date:** [February 14, 2018, 9:07am UTC](https://discuss.elastic.co/t/failed-to-connect-to-master-node/119754/1 "2018-02-14T09:07:22Z")

</div>

Hi,

We are having problems setting up a cluster on Azure using docker. We are using three docker run commands to start each of the nodes, along the lines of :-

`sudo docker run --net=bridge -d -e "node.master=true" -e "transport.host=0.0.0.0" -e "network.host=0.0.0.0" -e "network.bind_host=0.0.0.0" -e "network.publish_host=0.0.0.0" -e "node.name=esuk02" -e "discovery.zen.minimum_master_nodes=2" -e "ELASTIC_PASSWORD=xxx" -e "bootstrap.memory_lock=true" -e "ES_JAVA_OPTS=-Xms8g -Xmx8g" -e "xpack.security.http.ssl.enabled=true" -e "xpack.security.transport.ssl.enabled=true" -e "xpack.security.transport.ssl.verification_mode=certificate" -e "xpack.ssl.certificate_authorities=x-pack/certificates/certs/ca/ca.crt" -e "xpack.ssl.certificate=x-pack/certificates/certs/ukselastic2/ukselastic2.crt" -e "xpack.ssl.key=x-pack/certificates/certs/ukselastic2/ukselastic2.key" -e "discovery.zen.ping.unicast.hosts=10.4.0.10,10.4.0.11,10.4.0.12" --mount type=bind,source=/var/lib/waagent,target=/usr/share/elasticsearch/config/x-pack/certificates,readonly -p 9200:9200 -p 9300:9300 --ulimit memlock=-1:-1 docker.elastic.co/elasticsearch/elasticsearch-platinum:6.2.0`

This is created with Terraform, which substitutes the relevant items such as the node.name and the certs

In our logs we're seeing :-

> <https://gist.github.com/sbutt/5fc33c7de52d855501160cb2821d8d91>

which I think looks OK. But on node 2 :-

> <https://gist.github.com/sbutt/9452778e34e2ed8a683d011e7862f653>

And likewise we get a similar thing on node 3.

I can netcat to each of the nodes from each other using 9200 or 9300, using either the 10.4.0.x address OR the 172.17.0.x address, from the host or by executing a bash prompt within the docker container and that works just fine, so I don't think I have connectivity or firewall type problems. It must be something wrong with my ES config then (right?)

Here's the docker run for node 1 :-

```
sudo docker run --net=bridge -d -e "node.master=true" -e "transport.host=0.0.0.0" -e "network.host=0.0.0.0" -e "node.name=esuk01" -e "discovery.zen.minimum_master_nodes=2" -e "ELASTIC_PASSWORD=xxx" -e "bootstrap.memory_lock=true" -e "ES_JAVA_OPTS=-Xms8g -Xmx8g" -e "xpack.security.http.ssl.enabled=true" -e "xpack.security.transport.ssl.enabled=true" -e "xpack.security.transport.ssl.verification_mode=certificate" -e "xpack.ssl.certificate_authorities=x-pack/certificates/certs/ca/ca.crt" -e "xpack.ssl.certificate=x-pack/certificates/certs/ukselastic1/ukselastic1.crt" -e "xpack.ssl.key=x-pack/certificates/certs/ukselastic1/ukselastic1.key" -e "discovery.zen.ping.unicast.hosts=10.4.0.10,10.4.0.12" --mount type=bind,source=/var/lib/waagent,target=/usr/share/elasticsearch/config/x-pack/certificates,readonly -p 9200:9200 -p 9300:9300 --ulimit memlock=-1:-1 docker.elastic.co/elasticsearch/elasticsearch-platinum:6.2.0

```

and node 0 :-

`sudo docker run --net=bridge -d -e "node.master=true" -e "transport.host=0.0.0.0" -e "network.host=0.0.0.0" -e "node.name=esuk00" -e "discovery.zen.minimum_master_nodes=2" -e "ELASTIC_PASSWORD=xxx" -e "bootstrap.memory_lock=true" -e "ES_JAVA_OPTS=-Xms8g -Xmx8g" -e "xpack.security.http.ssl.enabled=true" -e "xpack.security.transport.ssl.enabled=true" -e "xpack.security.transport.ssl.verification_mode=certificate" -e "xpack.ssl.certificate_authorities=x-pack/certificates/certs/ca/ca.crt" -e "xpack.ssl.certificate=x-pack/certificates/certs/ukselastic0/ukselastic0.crt" -e "xpack.ssl.key=x-pack/certificates/certs/ukselastic0/ukselastic0.key" --mount type=bind,source=/var/lib/waagent,target=/usr/share/elasticsearch/config/x-pack/certificates,readonly -p 9200:9200 -p 9300:9300 --ulimit memlock=-1:-1 docker.elastic.co/elasticsearch/elasticsearch-platinum:6.2.0`

Many thanks!

Steve Button

---

<div class="post-metadata">

**Author:** ![sbu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbu/32/27718_2.png) [@sbu](https://discuss.elastic.co/u/sbu)\
**Post date:** [February 14, 2018, 9:28am UTC](https://discuss.elastic.co/t/failed-to-connect-to-master-node/119754/2 "2018-02-14T09:28:07Z")

</div>

Actually, that's not quite true. I can

nc -v 172.17.0.1 9200  
nc -v 172.17.0.1 9300  
nc -v 172.17.0.2 9200  
nc -v 172.17.0.2 9300

From inside the container and they all work OK, but 172.17.0.3 doesn't work. I guess I should change network.host or transport.host to bind to the 10.4.0.0 network instead of 0.0.0.0 which I guess is ANY interface. I'm a little unclear what each of these parameters actually does, and the docs don't expand on it much (at least the ones I was looking at)

---

<div class="post-metadata">

**Author:** ![sbu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sbu/32/27718_2.png) [@sbu](https://discuss.elastic.co/u/sbu)\
**Post date:** [February 15, 2018, 11:58am UTC](https://discuss.elastic.co/t/failed-to-connect-to-master-node/119754/3 "2018-02-15T11:58:06Z")

</div>

I'm still looking into this, and further to this I've found that if I set transport.host to the IP of the node (which is shared with the docker container) for example 10.4.0.11 then I get "BindException[Cannot assign requested address];" and it's complaining about ports 9300-9400, however when I set transport.host to 0.0.0.0 it keeps running, but I get the message :-

```
[2018-02-15T11:43:39,122][WARN][o.e.d.z.ZenDiscovery] [esuk01] failed to connect to master [{esuk00}{7AKsFpnVT2CazmBegPA4Lw}{W_f7IPanQc6PgXOqO5q0MA}{172.17.0.2}{172.17.0.2:9300}{ml.machine_memory=16797728768, ml.max_open_jobs=20, ml.enabled=etrying...
org.elasticsearch.transport.ConnectTransportException: [esuk00][172.17.0.2:9300] handshake failed. unexpected remote node {esuk01}{KllV7qs6R5WjFl0ZFqsvjA}{xiKH4kRKTOa9zIWbiQnEWA}{172.17.0.2}{172.17.0.2:9300}{ml.machine_memory=16797728768, ml.max_o20, ml.enabled=true}

```

which is interesting, because this is from node es01, and so it must be talking to es00 to get this message (otherwise how would it know if the existence of "es00"?), so I guess the question I need to answer is what does "failed to connect to master" actually mean? What makes that happen?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 15, 2018, 11:58am UTC](https://discuss.elastic.co/t/failed-to-connect-to-master-node/119754/4 "2018-03-15T11:58:08Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
