Invalid Certificate Errors when setting up SSL/HTTPS for Elasticsearch in Docker

[encompassJeff (Jeef)]

Hi there!

Some basic background: My client has recently requested that we transition to using ELK stack to power our search and metrics. We are running Laravel 5.8/PHP 7.3 on the backend (feeding a ReactJS front-end via API) and using a variation of the popular Laradock repo to run our containers with docker-compose.

I've been successful in setting up basic authentication with xpack security, but I also need to set up an SSL certificate and HTTPS. I've followed several guides in the official documentation and here in the forums, and have managed to get my certificates created and stored in the host, then mounted to a named volume in the elasticsearch container. However, if I try to access the instance via public url https://my-backend-url.com:9200 I get a NET::ERR_CERT_INVALID response rather than a secured connection that prompts me for the username and password authentication to view the instance details.

Can someone help me understand how to resolve this issue? I'm struggling to understand how to fix this because the certificates should be valid, as I'm following the documented process for creating them.

Here is my instances.yml file

instances:
  - name: elasticsearch
    dns:
      - elasticsearch
      - localhost
    ip:
      - 127.0.0.1

My create-certs.yml file

version: '2.2'

services:
  create_certs:
    container_name: create_certs
    image: docker.elastic.co/elasticsearch/elasticsearch:7.8.1
    command: >
      bash -c '
        if [[ ! -f /certs/bundle.zip ]]; then
          bin/elasticsearch-certutil cert --silent --pem --in config/certificates/instances.yml -out /certs/bundle.zip;
          unzip /certs/bundle.zip -d /certs;
        fi;
        chown -R 1000:0 /certs  '
    user: "0"
    working_dir: /usr/share/elasticsearch
    volumes:
      - ./certs:/certs
      - .:/usr/share/elasticsearch/config/certificates

volumes: {"certs"}

And my docker-compose.yml

version: '3'

networks:
  frontend:
    driver: ${NETWORKS_DRIVER}
  backend:
    driver: ${NETWORKS_DRIVER}

volumes:
  elasticsearch:
  certs:

services:
    elasticsearch:
      build: ./elasticsearch
      volumes:
        - elasticsearch:/usr/share/elasticsearch/data
        - ./certs:${CERTS_DIR}
      environment:
        - cluster.name=laradock-cluster
        - node.name=laradock-node
        - bootstrap.memory_lock=true
        - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
        - "ELASTIC_PASSWORD=${ELASTIC_PASSWORD}"
        - cluster.initial_master_nodes=laradock-node
        - xpack.security.enabled=true
        - xpack.security.http.ssl.enabled=true
        - xpack.security.http.ssl.key=${CERTS_DIR}/elasticsearch/elasticsearch.key
        - xpack.security.http.ssl.certificate_authorities=${CERTS_DIR}/ca/ca.crt
        - xpack.security.http.ssl.certificate=${CERTS_DIR}/elasticsearch/elasticsearch.crt
        - xpack.security.transport.ssl.enabled=true
        - xpack.security.transport.ssl.verification_mode=certificate 
        - xpack.security.transport.ssl.certificate_authorities=${CERTS_DIR}/ca/ca.crt
        - xpack.security.transport.ssl.certificate=${CERTS_DIR}/elasticsearch/elasticsearch.crt
        - xpack.security.transport.ssl.key=${CERTS_DIR}/elasticsearch/elasticsearch.key
      ulimits:
        memlock:
          soft: -1
          hard: -1
      ports:
        - "${ELASTICSEARCH_HOST_HTTP_PORT}:9200"
        - "${ELASTICSEARCH_HOST_TRANSPORT_PORT}:9300"
      depends_on:
        - php-fpm
      networks:
        - frontend
        - backend

The container builds and stays up when running docker-compose up -d elasticsearch but I cannot seem to remove the invalid certificate error when I try to access the instance on port 9200.

I'm a bit green to ELK stack and how to fit it in with our ecosystem, so possibly there is some configuration issue causing this that I'm unaware of. Just to add a bit more of an info dump about our setup: Currently I have SSL certificates set up for https://my-backend-url.com which is the host of endpoints accessed by our front-end application. However, when I initially set up elasticsearch with basic authentication, I had to access the instance at the non-secure url http://my-backend-url.com in order to be prompted to input the auth credentials, regardless of having SSL for the rest of the backend application.

Is it possible that this invalid certificate error is because I am attempting to set up 2 different sets of certs for (kind of) the same host? I figured things would be siloed due to access elasticsearch on port 9200 but maybe I'm wrong there? Should I be mounting the SSL certificates I'm using for nginx also for the elasticsearch SSL?

Obviously I have a lot of questions, and probably more coming. If anyone can offer help in understanding where my issue is coming from, I'd appreciate it.