Variables substitution doesn't work from keystore in username/password fields of metricbeat.yml in autodiscover.providers.templates.config section

Hi there!

How can I use keystore variables in autodiscover.providers.templates.config section of metricbeat.yml? It looks like broken. Version 7.1.0.

How to reproduce:

  1. Setup Docker for running next services
  2. Setup Elasticsearch and configure Metricbeat to connect to it
  3. Run a RabbitMQ service with an username and a password for authorization
  4. Add QUEUE_USERNAME and QUEUE_PASSWORD to Metricbeat keystore via metricbeat keystore add with appropriate values
  5. Add a block like this to metricbeat.yml:
  autodiscover:
    providers:
      - type: docker
        templates:
          - condition:
              equals:
                docker.container.labels.com.docker.swarm.service.name: depth-history_queue
            config:
              - module: rabbitmq
                period: 10s
                hosts: ['${data.host}:${data.port}']
                username: ${QUEUE_USERNAME}
                password: ${QUEUE_PASSWORD}
  1. Start the beat

Expected:
Metricbeat successfully connected to RabbitMQ service.

But it doesn't.

Interesting additional info:

  1. For debugging purposes, you can use "${QUEUE_PASSWORD}" as a value for field output.elasticsearch.hosts of metricbeat.yml and you will see the finish value of QUEUE_PASSWORD variable from the keystore in logs of the beat (in error messages about the connection to Elasticsearch). Therefore the keystore really store the correct value.

  2. If you replace ${QUEUE_USERNAME} and ${QUEUE_PASSWORD} by hard-coded values in metricbeat.yml and restart the beat it will work. Therefore Metricbeat can connect to RabbitMQ with auth.

  3. Even if you return the vars back to metricbeat.yml and add 2 Linux env vars QUEUE_USERNAME and QUEUE_PASSWORD and restart the beat it will work too. Therefore some variables substitution works in the autodiscover.providers.templates.config section.

Hi @nevmerzhitsky,

Thank you for your feedback. Just want to make sure I understood this, did you test this very same config without the keystore references? If the answer is yes that would probably mean there is an issue when using keystore variables from autodiscover templates. Could you open a bug report for this? https://github.com/elastic/beats/issues

Best regards

Yes, I did it - mentioned as point 2 in the last list in the post.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.