I am trying to enable SSL/TLS for my elasticsearch cluster using docker and docker-compose. My setup is 3 seperate machines all with a single elasticsearch node running on them. For the certificates I created them on a single volume and copied that volume to all 3 machines. I used Encrypting communications in an Elasticsearch Docker Container | Elasticsearch Guide [7.12] | Elastic as a guide of the settings to enable for SSL/TLS.
When running docker-compose my nodes are unable to discover each other and am receiving the following error:
es01 | {"type": "server", "timestamp": "2021-04-27T18:46:31,221Z", "level": "WARN", "component": "o.e.c.c.ClusterFormationFailureHelper", "cluster.name": "docker-cluster", "node.name": "es01", "message": "master not discovered or elected yet, an election requires 2 nodes with ids [N6koZEyeR0-VketSDFUOEA, vLwJcl13TSyTXlX2CN4QHQ], have discovered [{es01}{vLwJcl13TSyTXlX2CN4QHQ}{GDvFih09QrC2OkZ6NA0P9g}{172.24.5.171}{172.24.5.171:9300}{cdfhilmrstw} ml.machine_memory=4143259648, xpack.installed=true, transform.node=true, ml.max_open_jobs=20, ml.max_jvm_size=536870912}] which is not a quorum discovery will continue using [172.24.5.172:9300, 172.24.5.188:9300] from hosts providers and [{es01}{vLwJcl13TSyTXlX2CN4QHQ}{GDvFih09QrC2OkZ6NA0P9g}{172.24.5.171}{172.24.5.171:9300}{cdfhilmrstw}{ml.machine_memory=4143259648, xpack.installed=true, transform.node=true, ml.max_open_jobs=20, ml.max_jvm_size=536870912}] from last-known cluster state; node term 117, last-accepted version 0 in term 0" }
My .yml files look like this (names and ips are changed to match the correct machine and have matching environmental variables):
version: '2.2'
services:
es03:
container_name: es03
image: docker.elastic.co/elasticsearch/elasticsearch:7.12.0
environment:
- node.name=es03
- cluster.initial_master_nodes=172.24.5.171,172.24.5.172,172.24.5.188
- discovery.seed_hosts=172.24.5.188,172.24.5.172
- network.publish_host=172.24.5.171
- ELASTIC_PASSWORD=$ELASTIC_PASSWORD
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- xpack.license.self_generated.type=trial
- xpack.security.enabled=true
- xpack.security.http.ssl.enabled=true
- xpack.security.http.ssl.key=$CERTS_DIR/es03/es03.key
- xpack.security.http.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
- xpack.security.http.ssl.certificate=$CERTS_DIR/es03/es03.crt
- xpack.security.transport.ssl.enabled=true
- xpack.security.transport.ssl.verification_mode=certificate
- xpack.security.transport.ssl.certificate_authorities=$CERTS_DIR/ca/ca.crt
- xpack.security.transport.ssl.certificate=$CERTS_DIR/es03/es03.crt
- xpack.security.transport.ssl.key=$CERTS_DIR/es03/es03.key
volumes: ['data03:/usr/share/elasticsearch/data', 'certs:$CERTS_DIR']
ports:
- 9200:9200
healthcheck:
test: curl --cacert $CERTS_DIR/ca/ca.crt -s https://localhost:9200 >/dev/null; if [[ $$? == 52 ]]; then echo 0; else echo 1; fi
interval: 30s
timeout: 10s
retries: 5
volumes: {"data03", "certs"}
Before adding security my nodes were clustering properly with only the first 4 yml environment options. Ports 9200 and 9300 are open and can be pinged from all machines.