Filebeat autodiscover namespace_defaults not work

Elastic Stack Beats
,
1

Hello.
I'm using Elasticsearch filebeat 7.17.

According to the document, filebeat can use namespace's annotation when pod's annotation does not exist.

But it seems to be working against my expectations.

The following cases operate normally.

...
apiVersion: v1
kind: Pod
metadata:
  annotations:
    log-topic: service
...

However, it does not work when you delete the comment from pod and put the same annotation in namespace.

This is my filebeat setting file.

filebeat.autodiscover:
  providers:
  - type: kubernetes
    hint.enabled: true
    add_resource_metadata:
      namespace:
        include_annotations: ["log-topic"]
    templates:
    - condition.contains:
        kubernetes.annotations.log-topic: service
      config:
      - type: docker
        containers.ids:
          - ${data.kubernetes.container.id}
        fields:
          log_topic: k8s.logging.filebeat.service
output.kafka:
  hosts: ["kafka_fqdn:port"]
  topic: '%{[fields.log_topic]}'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
  worker: 6
2

Hi @lcc3108,

Seems here is an issue with documentation - here is an original PR for this feature, this comment as well as tests explains what should be expected.
I will create an issue to adjust documentation.

3

I will try the contents of the link.
And I hope the issue is created and the document is updated.

Thank you.

4

It still doesn't work.

filebeat.autodiscover:
  providers:
  - type: kubernetes
    hint.enabled: true
    add_resource_metadata:
      namespace:
        enabled: true
        include_annotations: ["log-topic"] # comment or not comment tested

    templates:
    - condition.contains:
        kubernetes.annotations.log-topic: service
      config:
      - type: docker
        containers.ids:
          - ${data.kubernetes.container.id}
        fields:
          log_topic: k8s.logging.filebeat.service
output.kafka:
  hosts: ["kafka-confluent-1-cp-kafka-headless.kafka.svc.cluster.local:9092"]
  topic: '%{[fields.log_topic]}'
  partition.round_robin:
    reachable_only: false
  required_acks: 1
  compression: gzip
  max_message_bytes: 1000000
  worker: 6

$ kubectl describe ns default
Name:         default
Labels:       log-type=service
Annotations:  log-topic: service
Status:       Active
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.