External access to ElasticSearch cluster

[bobus (George)]

Using Install Elasticsearch with Docker | Elasticsearch Guide [8.11] | Elastic I can easily setup a 3-node cluster on a single host, with docker-compose.

This is all fine if I just use ES via the included Kibana container. However I need to access this cluster from external hosts. This becomes problematic because the nodes inside the cluster are exposed through their docker-internal IP address. The application uses this API call below to get the addresses, and then of course errors out.

$ curl 172.16.0.146:9200/_nodes/http?pretty
{
"_nodes" : {
"total" : 3,
"successful" : 3,
"failed" : 0
},
"cluster_name" : "es-cluster-test",
"nodes" : {
"hYCGiuBLQMK4vn5I3C3pQQ" : {
"name" : "es01",
"transport_address" : "192.168.48.3:9300",
"host" : "192.168.48.3",
"ip" : "192.168.48.3",
"version" : "8.2.2",
.....

How can I overcome this? I have tried exposing the 9200/9300 ports for all 3 nodes to different ports on the docker-host, and then adding a network.publish_host=172.16.0.146 environment setting to each node, but this results in three 1-node clusters.

Someone must have faced this one in the past... It looks like a very common scenario...

Thanks for any clues.

[warkolm (Mark Walkom)]

Welcome to our community!

Try setting network.host: 0.0.0.0 to bind Elasticsearch to all interfaces.

[bobus (George)]

Thanks for the welcome!

I set that as an environment variable in all 3 instances, and the only thing that changed in the output of the curl is the internal IP addresses that docker chose to assign to the containers.

My starting point was the vanilla yaml that is provided in the link I list above, i.e. without my two additional settings for network.publish_host and the port mapping of the 6 internal ports to the docker host.

es01:
depends_on:
setup:
condition: service_healthy
image: docker.elastic.co/elasticsearch/elasticsearch:${STACK_VERSION}
volumes:
- certs:/usr/share/elasticsearch/config/certs
- /data/esdata01:/usr/share/elasticsearch/data
ports:
- ${ES_PORT}:9200
environment:
- network.host=0.0.0.0
- node.name=es01
- cluster.name=${CLUSTER_NAME}
- cluster.initial_master_nodes=es01,es02,es03
- discovery.seed_hosts=es02,es03

I think my use-case scenario is a routine one, figuring it out would help lots of people....