Environment variable for setting logstash CA file for xpack monitoring

I am trying to setup Docker Logstash with xpack monitoring, however I am not able to get the environment variable for specifying the CA file location.
xpack.monitoring.elasticsearch.ssl.certificate_authority

Here is the docker-compose file for Logstash

version: "3.7"
services:
  logstash01:
    image: docker.elastic.co/logstash/logstash:7.2.1
    container_name: logstash01
    ports:
      - "11000-11100:11000-11100"
      - "11000-11100:11000-11100/udp"
    networks:
      - logstash
    environment:
      NODE_NAME: logstash01
      CONFIG_RELOAD_AUTOMATIC: "true"
      QUEUE_TYPE: persisted
      QUEUE_MAX_BYTES: 2gb
      XPACK_MONITORING_ENABLED: "true"
      XPACK_MONITORING_ELASTICSEARCH_USERNAME: logstash_system
      XPACK_MONITORING_ELASTICSEARCH_PASSWORD: <snip>
      XPACK_MONITORING_ELASTICSEARCH_HOSTS: https://elasticsearch01:9200
      XPACK_MONITORING_ELASTICSEARCH_SSL_CERTIFICATEAUTHORITY: /usr/share/logstash/certs/cert.crt
    volumes:
      - type: volume
        source: logstash_cache
        target: /usr/share/logstash/data/queue
        volume:
          nocopy: true
      - type: volume
        source: logstash_pipelines
        target: /usr/share/logstash/pipeline
        volume:
          nocopy: true
      - type: volume
        source: logstash_certs
        target: /usr/share/logstash/certs
        volume:
          nocopy: true
    logging:
      driver: "json-file"
      options:
        max-size: "5m"
        max-file: "1"
    restart: unless-stopped
networks:
  logstash:
    driver: bridge
    ipam:
      config:
        - subnet: 10.111.2.0/24
volumes:
  logstash_cache:
    driver_opts:
      type: nfs
      o: "addr=10.10.25.1,hard,tcp,nolock,sync,rw"
      device: ":/mnt/data/logstash/persist_cache"
  logstash_pipelines:
    driver_opts:
      type: nfs
      o: "addr=10.10.25.1,hard,tcp,nolock,sync,rw"
      device: ":/mnt/data/logstash/pipeline"
  logstash_certs:
    driver_opts:
      type: nfs
      o: "addr=10.10.25.1,hard,tcp,nolock,sync,rw"
      device: ":/mnt/data/logstash/certs"

When I enter the container and look at the generated yml file I do not see the certificate_authority value at all:

cat config/logstash.yml
config.reload.automatic: true
http.host: 0.0.0.0
node.name: logstash01
queue.max_bytes: 2gb
queue.type: persisted
xpack.monitoring.elasticsearch.hosts: https://elasticsearch01:9200
xpack.monitoring.elasticsearch.password: <snip>
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.enabled: true

Any help would be appreciated as this has been bothering me for a few days now with no luck on finding a solution.

@jarpy I think you might have worked on this Docker image and env2yaml at some point. Does anything come to mind here?

I was able to get this to work by using the following docker-compose file:

version: "3.7"
services:
  logstash01:
    image: docker.elastic.co/logstash/logstash:7.2.1
    container_name: logstash01
    ports:
      - "11000-11100:11000-11100"
      - "11000-11100:11000-11100/udp"
    networks:
      - logstash
    environment:
      NODE_NAME: logstash01
      CONFIG_RELOAD_AUTOMATIC: "true"
      QUEUE_TYPE: persisted
      QUEUE_MAX_BYTES: 2gb
      XPACK_MONITORING_ENABLED: "true"
      XPACK_MONITORING_ELASTICSEARCH_USERNAME: logstash_system
      XPACK_MONITORING_ELASTICSEARCH_PASSWORD: <snip>
      XPACK_MONITORING_ELASTICSEARCH_HOSTS: https://elasticsearch01:9200
      xpack.monitoring.elasticsearch.ssl.certificate_authority: /usr/share/logstash/certs/cert.crt
    volumes:
      - type: volume
        source: logstash_cache
        target: /usr/share/logstash/data/queue
        volume:
          nocopy: true
      - type: volume
        source: logstash_pipelines
        target: /usr/share/logstash/pipeline
        volume:
          nocopy: true
      - type: volume
        source: logstash_certs
        target: /usr/share/logstash/certs
        volume:
          nocopy: true
    logging:
      driver: "json-file"
      options:
        max-size: "5m"
        max-file: "1"
    restart: unless-stopped
networks:
  logstash:
    driver: bridge
    ipam:
      config:
        - subnet: 10.111.2.0/24
volumes:
  logstash_cache:
    driver_opts:
      type: nfs
      o: "addr=10.10.25.1,hard,tcp,nolock,sync,rw"
      device: ":/mnt/data/logstash/persist_cache"
  logstash_pipelines:
    driver_opts:
      type: nfs
      o: "addr=10.10.25.1,hard,tcp,nolock,sync,rw"
      device: ":/mnt/data/logstash/pipeline"
  logstash_certs:
    driver_opts:
      type: nfs
      o: "addr=10.10.25.1,hard,tcp,nolock,sync,rw"
      device: ":/mnt/data/logstash/certs"

I am not sure why the environment variable does not work as explained in the docs, but this way appears to work correctly.

Sorry for the slow response. It looks like the initial version was missing the last underscore. It seems to work if you add it in:

$ docker run --rm -it \
  -e XPACK_MONITORING_ELASTICSEARCH_SSL_CERTIFICATE_AUTHORITY=canary \
  docker.elastic.co/logstash/logstash:7.1.1 \
  cat /usr/share/logstash/config/logstash.yml | grep canary

xpack.monitoring.elasticsearch.ssl.certificate_authority: canary

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.