# Filebeat Autodiscover appenders - configuration not applied

**URL:** <https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [October 19, 2020, 5:25pm UTC](https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588 "2020-10-19T17:25:42Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![solarwinds](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/solarwinds/32/37390_2.png) [@solarwinds](https://discuss.elastic.co/u/solarwinds)\
**Post date:** [October 19, 2020, 5:25pm UTC](https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588/1 "2020-10-19T17:25:42Z")

</div>

Hi everyone, before opening an issue on Github, I will try here. As subject states, Filebeat Autodiscover appenders are not working as expected - configuration is not being applied. If I configure `add_docker_metadata` as label on service, Docker Labels are being applied, using autodiscover plugin config hints.

I am running ELK stack on Mac, using Docker Desktop for Mac.  
Filebeat version is **7.9.1**.  
Anyhow, here goes my minimal filebeat.yml config:

```auto
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
  
  
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
      hints.default_config:
        type: container
        paths: 
          - '/var/lib/docker/containers/${data.container.id}/*.log'
        combine_partial: false
        fields_under_root: true
        exclude_files: [".gz$", "_[0-9]{4}-[0-9]{2}-[0-9]{2}"]
        tail_files: false
        ignore_older: 24h
        close_inactive: 1h
      appenders:
        - type: config
          merge: append
          config:
            - add_docker_metadata:
                labels.dedot: false

output.elasticsearch:
  hosts: 'http://es01:9200'
  pipeline: "filebeat-7.9.1-pipeline"
  

```

For testing purpose, I am using a `docker-compose` to spin up another service:

```auto
version: '3.2'
services: 
  docker-logs: 
    user: root
    image: docker-logs:latest
    container_name: docker-logs
    labels:
      # co.elastic.logs/processors.add_docker_metadata.labels.dedot: "false"
      co.elastic.logs/processors.add_fields.fields.name: "MY_FIELDS"
      co.elastic.logs/processors.add_fields.fields.id: "MY_VAULE"

```

However, the Docker labels are only being applied, if I configure the `add_docker_metadata` on service above, where desired would be to apply the `add_docker_metadata` processor in `autodiscover` context of filebeat.yml configuration file.

From Filebeat logs there is only one line being logged, relevant to appenders:

```auto
filebeat | 2020-10-19T17:18:53.562Z	INFO	cfgfile/reload.go:224	Loading of config files completed.
filebeat | 2020-10-19T17:18:53.577Z	WARN	[cfgwarn]	config/config.go:50	BETA: The config appender is beta
filebeat | 2020-10-19T17:18:53.583Z	INFO	[autodiscover]	autodiscover/autodiscover.go:113	Starting autodiscover manager

```

EDITS: Mostly code formatting.

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [October 19, 2020, 8:47pm UTC](https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588/2 "2020-10-19T20:47:21Z")

</div>

If I understand your use case correctly, I think you can just use ["regular" autodiscover](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html) and don't actually need [hints-based autodiscover](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover-hints.html).

---

<div class="post-metadata">

**Author:** ![solarwinds](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/solarwinds/32/37390_2.png) [@solarwinds](https://discuss.elastic.co/u/solarwinds)\
**Post date:** [October 20, 2020, 10:10am UTC](https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588/3 "2020-10-20T10:10:25Z")

</div>

I would disagree. At least from the documentation perspective. What if I want to add some fields using `add_fields` in context of `autodiscover`?

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [October 20, 2020, 10:35am UTC](https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588/4 "2020-10-20T10:35:03Z")

</div>

Ah, I see, you want the additional fields to vary dynamically per service. Understood now. In that case, yes, you'll want hints-based autodiscover.

Let me try to reproduce your situation locally and get back to you.

---

<div class="post-metadata">

**Author:** ![solarwinds](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/solarwinds/32/37390_2.png) [@solarwinds](https://discuss.elastic.co/u/solarwinds)\
**Post date:** [October 20, 2020, 10:53am UTC](https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588/5 "2020-10-20T10:53:33Z")

</div>

Ideally yes, but also it should be applicable in case that condition isn't being specified.

Let me know if you need more details, but it should be pretty straightforward to reproduce the issue.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 17, 2020, 12:53pm UTC](https://discuss.elastic.co/t/filebeat-autodiscover-appenders-configuration-not-applied/252588/6 "2020-11-17T12:53:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.