Cannot cluster Elasticsearch when using Docker bridge network

Elastic Stack Elasticsearch
1

I have elasticserach master running on 2 hosts (10.123.4.10 & 10.123.4.11).
But they cannot see each other when using the Docker bridge network.

My docker-compose.yml:

version: '2'
services:

  elasticsearch:
    image: elasticsearch:2.3.0
    ports:
        - "9200:9200"
    environment:
        - HOST_HOSTNAME=${HOST_HOSTNAME}
    volumes:
      - ${PWD}/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
    restart: always
    networks:
      - es

networks:
   es:
     driver: bridge

networks:
es:
driver: bridge`

My elasticsearch.yml files:

node1

cluster.name: "es-cluster"
node.name: ${HOST_HOSTNAME}
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.multicast.enabled: false
network.bind_host: 0.0.0.0
network.publish_host: 10.123.4.10
discovery.zen.ping.unicast.hosts: 10.123.4.10, 10.123.4.11

node2

cluster.name: "es-cluster"
node.name: ${HOST_HOSTNAME}
discovery.zen.minimum_master_nodes: 2
discovery.zen.ping.multicast.enabled: false
network.bind_host: 0.0.0.0
network.publish_host: 10.123.4.11
discovery.zen.ping.unicast.hosts: 10.123.4.10, 10.123.4.11

When I switch from bridge to network_mode: "host" the nodes can see each other

Elasticsearch Docker image version 2.3.0
Docker version 1.11.1, build 5604cbe
docker-compose version 1.7.0, build 0d7bf73
Docker hosts running Ubuntu 14.04.4 LTS

2

Hi,

When you put these containers on your es network, what is the ouput of following command ?

docker network inspect es

3 
[
    {
        "Name": "es",
        "Id": "6e692ea87a53ead3b8651b9661bc1812125c6b0572351a5657b4dedda02feb2e",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": null,
            "Config": [
                {
                    "Subnet": "172.21.0.0/16",
                    "Gateway": "172.21.0.1/16"
                }
            ]
        },
        "Internal": false,
        "Containers": {
            "eb37d9b2b1a58f51dbe298d288822f7c8ceb30b69636956d4aa01ddfb64aa776": {
                "Name": "elasticsearch",
                "EndpointID": "27567d038c484f58e040be10c60b6cace60740189f76cc10486e292f7c817716",
                "MacAddress": "02:42:ac:15:00:02",
                "IPv4Address": "172.21.0.2/16",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]
4

OK, I misread your 1st post, I thought the two containers were running on the same host.

Anyway, as mentioned on the Docker documentation, host networking consists in running the container in the same network configuration as the host. ES nodes can recognize each other because they have the same IP as your host and as your ES configuration files.

However, with a bridge network, it is kind of NATted. So when you specify for each ES node (in elasticsearch.yml) its IP as 10.123.4.xxx, it won't recognize itself, because its real IP address is 172.21.0.xxx

I don't know if my explanations are good and understandable, but I my guess is that you should either keep on using host network, either use overlay network with docker swarm.

1 Like
5

Ya I figured it would still work. Since I'm using network.bind_host and network.publish_host

I should of added that I've followed this wiki:

6

Did you also try with the default bridge network provided by Docker ?
Because maybe this option would help NATting communications between the two containers :

"com.docker.network.bridge.enable_ip_masquerade": "true",

If this work, maybe you could try creating your custom bridge network with this option enabled too ?

7

Ya still same issue. I'll stick with network_mode: "host" for now