Autodiscover Kubernetes + annotations on pods not working as excepted

[Roman_Kournjaev]

I am trying to make filebeat work with the official elastic helm chart.
I would like to parse only the pods that have the "logging" : "json_log" annotation. As soon as i deploy pods that have that annotation for some reason i am getting all the events parsed , like the logs from filebeats. What exactly i am configuring wrong?

   filebeat.autodiscover:
  providers:
    - type: kubernetes
      in_cluster: true
      tags:
        - "kubernetes"
      templates:
        - condition:
            contains:
              kubernetes.annotations.logging: "json_log"
          config:
            - type: container
              json.keys_under_root: true
              json.add_error_key: true
              json.message_key: msg
              paths:
                - '/var/lib/docker/containers/*/*.log'

processors:
  - add_kubernetes_metadata:
      in_cluster: true

output.elasticsearch:
  hosts: ["http://xxxxx:9200"]

[pmercado (PM)]

Hi @Roman_Kournjaev,

when the condition is met the configuration under that condition is triggered. That configuration is reading all logs from the folder, which includes all containers managed by docker.

Maybe using some available variable like ${data.kubernetes.container.id} would fix it.

Here you can find some info and an explicit warning for that scenario: https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html