Filebeat autodiscover namespace_defaults not work

[lcc3108]

Hello.
I'm using Elasticsearch filebeat 7.17.

According to the document, filebeat can use namespace's annotation when pod's annotation does not exist.

Hints based autodiscover | Filebeat Reference [7.17] | Elastic

But it seems to be working against my expectations.

The following cases operate normally.

...
apiVersion: v1
kind: Pod
metadata:
  annotations:
    log-topic: service
...

However, it does not work when you delete the comment from pod and put the same annotation in namespace.

This is my filebeat setting file.

filebeat.autodiscover:
  providers:
  - type: kubernetes
    hint.enabled: true
    add_resource_metadata:
      namespace:
        include_annotations: ["log-topic"]
    templates:
    - condition.contains:
        kubernetes.annotations.log-topic: service
      config:
      - type: docker
        containers.ids:
          - ${data.kubernetes.container.id}
        fields:
          log_topic: k8s.logging.filebeat.service
output.kafka:
  hosts: ["kafka_fqdn:port"]
  topic: '%{[fields.log_topic]}'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
  worker: 6

[Tetiana_Kravchenko]

Hi @lcc3108,

Seems here is an issue with documentation - here is an original PR for this feature, this comment as well as tests explains what should be expected.
I will create an issue to adjust documentation.

[lcc3108]

I will try the contents of the link.
And I hope the issue is created and the document is updated.

Thank you.

[lcc3108]

It still doesn't work.

filebeat.autodiscover:
  providers:
  - type: kubernetes
    hint.enabled: true
    add_resource_metadata:
      namespace:
        enabled: true
        include_annotations: ["log-topic"] # comment or not comment tested

    templates:
    - condition.contains:
        kubernetes.annotations.log-topic: service
      config:
      - type: docker
        containers.ids:
          - ${data.kubernetes.container.id}
        fields:
          log_topic: k8s.logging.filebeat.service
output.kafka:
  hosts: ["kafka-confluent-1-cp-kafka-headless.kafka.svc.cluster.local:9092"]
  topic: '%{[fields.log_topic]}'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
  worker: 6

$ kubectl describe ns default
Name:         default
Labels:       log-type=service
Annotations:  log-topic: service
Status:       Active