Dedot Kubernetes labels and annotations


(Anton Johansson) #1

I'm using Filebeat with Kubernetes autodiscovery. I'm using version 6.5.3 for all Elastic components. I've had some struggle with Kubernetes labels and annotations which can contain dots and slashes. I know that this pull request is merged and will be included in 6.7 and 7.0 as soon as they are ready.

This solves the issue with how these labels and annotations end up in Elasticsearch. However, I've also had an issue conditionals in my autodiscovery, that checks a specific annotation. Here is the configuration that I would like to have:

logging:
  level: warning
filebeat:
  autodiscover:
    providers:
      - type: kubernetes
        include_annotations: ['logging.example.com/harvest']
        templates:
          - condition:
              not:
                equals:
                  kubernetes.annotations.logging_example_com/harvest: 'false'
            config:
              - type: log
                paths:
                  - '/var/log/containers/${data.kubernetes.pod.name}_${data.kubernetes.namespace}_${data.kubernetes.container.name}-${data.kubernetes.container.id}.log'
                symlinks: true
                multiline:
                  pattern: '^\d{4}-\d{2}-\d{2}'
                  negate: true
                  match: after
                document_type: kube-logs
  config:
    modules:
      reload:
        enabled: false
output:
  logstash:
    hosts:
      - 'logstash:5044'

This does not work, so I've temporarily changed the annotation name to just "harvest_logs", which seems to work flawlessly. But I was expecting the full annotation name to work, because this pull request was merged a long time ago and is included in the version that I'm using.

What's the difference between the two pull requests? Which one fixes what? Can I expect the above configuration to work as I expect it to once the change from the PR is released?

Another question is if I've handled the slash properly. I assume the slash works as it is, and shouldn't be dedotted anywhere?

Also, I know there aren't any release dates for 6.7 or 7.0, but does anyone have a clue how long I should be expected to wait? Roughly? :smiley:

I can't quite wrap my head around this. Any help appreciated!


(system) closed #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.