Elasticsearch, docker and iptables


Just installed elasticsearch container: https://hub.docker.com/_/elasticsearch/

On 4 RedHat entr Linux 7 VMs.... with a config like so:

cluster.name: Aves
node.name: "aves-01"
path.logs: /var/log/elasticsearch
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.timeout: 3s
discovery.zen.minimum_master_nodes: 3
discovery.zen.ping.unicast.hosts: [


When I start the containers... it seems to work fine (kopf plugin shows the cluster state as being good), but, all instances constantly log network connection errors like so:

[2015-08-22 17:44:20,006][WARN ][transport.netty          ] [aves-01] exception caught on transport layer [[id: 0xfda51f31]], closing connection
java.net.NoRouteToHostException: No route to host
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
        at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
        at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
        at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
        at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

and tcpdump shows constant reject like so:

listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
13:58:52.095383 IP > ICMP host unreachable - admin prohibited, length 68
13:58:52.095387 IP > ICMP host unreachable - admin prohibited, length 68
13:58:52.095621 IP > ICMP host unreachable - admin prohibited, length 68
13:58:52.095623 IP > ICMP host unreachable - admin prohibited, length 68

As if iptables is rejecting traffic from the container to itself? being the host IP, and the container IP. the icmp packet details show this is for tcp port 9300.

Whats going on?

Thank you so much.

This is a docker issue, not an Elasticsearch one. From what I have seen this sort of this is pretty common for any docker container.