# Can filebeat.autodiscover load external configuration files, like inputs and modules can?

**URL:** <https://discuss.elastic.co/t/can-filebeat-autodiscover-load-external-configuration-files-like-inputs-and-modules-can/254546>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 6, 2020, 3:16pm UTC](https://discuss.elastic.co/t/can-filebeat-autodiscover-load-external-configuration-files-like-inputs-and-modules-can/254546 "2020-11-06T15:16:01Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [November 6, 2020, 3:16pm UTC](https://discuss.elastic.co/t/can-filebeat-autodiscover-load-external-configuration-files-like-inputs-and-modules-can/254546/1 "2020-11-06T15:16:01Z")

</div>

Title says it all.

Filebeat has the capability to load external configuration files for inputs and modules, which is documented [here](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration-reloading.html).

Does this capability also exist for autodiscover configurations? I suspect the answer is "no," but want to confirm.

I tested it with the Elastic-provided Filebeat helm chart and the following configuration:

```auto
---
filebeat:
  autodiscover:
    providers:
      path: '${path.config}/provider-*.yml'
      reload:
        enabled: false
output:
  ...

```

and ensured that the provider files were in the correct place in the container. Filebeat started normally, and without errors, but did not load the provider files.

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [November 6, 2020, 4:17pm UTC](https://discuss.elastic.co/t/can-filebeat-autodiscover-load-external-configuration-files-like-inputs-and-modules-can/254546/2 "2020-11-06T16:17:21Z")

</div>

Hey @DougR,

You are right, this is not possible. We recently discussed about this here [https://github.com/elastic/beats/issues/22067](https://github.com/elastic/beats/issues/22067), and in principle we are not going to implement this.

What would be your use case?

---

<div class="post-metadata">

**Author:** ![DougR](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dougr/32/48095_2.png) [@DougR](https://discuss.elastic.co/u/DougR)\
**Post date:** [November 6, 2020, 5:12pm UTC](https://discuss.elastic.co/t/can-filebeat-autodiscover-load-external-configuration-files-like-inputs-and-modules-can/254546/3 "2020-11-06T17:12:40Z")

</div>

Unless I misread it, I'm not quite certain these are discussing the same thing.

The primary use case is that it simplifies maintenance. I use the Elastic-provided helm chart to deploy my filebeat as a daemonset to my Kubernetes cluster. We have numerous applications running on the same cluster, in separate namespaces. Currently, it is necessary to maintain a single `filebeat.yml` with all providers. This will allow me to maintain them as individual files, making them more manageable, similar to modules, and inputs. I.e., in my chart values for my filebeat, I would be able to include the following

```auto
---
filebeatConfig:
  filebeat.yml: |
    ---
    filebeat:
      autodiscover:
        providers:
          path: '${path.config}/provider-*.yml'
          reload:
            enabled: false
    output:
      ...

  provider-001.yml: |
    ...

  provider-002.yml: |
    ...

  provider-003.yml: |
    ...

```

and ensured that the provider files were in the correct place in the container. Filebeat started normally, and without errors, but did not load the provider files.

When using Ansible to manage the deployment, it allows me to load the provider files into an ansible `dict` object and do the following with a `filebeat.yml.j2` template:

```auto
filebeatConfig:
  filebeat.yml: |
    ---
    filebeat:
    autodiscover:
      providers:
        path: '${path.config}/provider-*.yml'
        reload:
          enabled: false
    output:
      ...

{% for item in filebeat_providers.keys()|default([])|sort %}
  {{ item }}: |
    {{ filebeat_providers[item]|indent(width=4) }}

{% endfor %}

```

While it is possible to insert them into the `filebeat.yml` in a similar fashion, which is what I currently do, allowing them to be read in as separate files is more fool-proof and easier to troubleshoot.

While I haven't looked at it before now, I think that hint-based configuration may be a good future solution for us. However, we are still early enough in adopting our strategy that I don't know that it's feasible at the moment.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 4, 2020, 7:12pm UTC](https://discuss.elastic.co/t/can-filebeat-autodiscover-load-external-configuration-files-like-inputs-and-modules-can/254546/4 "2020-12-04T19:12:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.