Load external configuration files with autodiscover

[hamid.haghshenas]

I'm going to monitor Docker Swarm containers using Metricbeat, which is itself a Swarm service. I've heard that I should use autodiscover instead of using modules directly.

Say I want to monitor services running uWSGI and PHP-FPM. Is it required to copy all contents of modules.d/uwsgi.yml and modules.d/php_fpm.yml in metricbeat.yml like below?

metricbeat.autodiscover:
  providers:
    - type: docker
      templates:
        - condition:
            contains:
              docker.container.image: my_python_app
          config:
            # Paste contents of uwsgi.yml here
        - condition:
            contains:
              docker.container.image: my_php_app
          config:
            # Paste contents of php_fpm.yml here

But this seems to end in a large metricbeat.yml file. Is there a way to use separate config files with autodiscover, like when we used modules directly?

[ruflin]

Currently you need all the configs in this file. One solution to solve this is using hint based autodiscovery: https://www.elastic.co/guide/en/beats/metricbeat/current/configuration-autodiscover-hints.html But to be honest I haven't tried of this works on Docker Swarm.

[hamid.haghshenas]

To use this solution, I need to config my Metricbeat by labeling Docker containers, right?

The idea itself is not bad and can be interpreted as annotating containers for metering.

Thanks again.

[ruflin]

Exactly.