Filebeat kubernetes autodiscover Multiple conditions

Elastic Stack Beats
1

The problem is When the condition has “kubernetes.labels.log-index” ，fields:log_topic: 'labels-{data.kubernetes.labels.log-index}', otherwise fields: log_topic: 'aip-{data.kubernetes.namespace}-${data.kubernetes.pod.name}'
The default condition not.equals.kubernetes.container.name: "filebeat" 。

Test 1
use filebeat processors

  • drop_event of “when” can use multiple conditions ？

Such as

 filebeat.autodiscover:
    providers:
      - type: kubernetes
        hints.enabled: true
        templates:
        processors:
        - drop_event:
            when:
              and:
                - not:
                    has_fields: ['kubernetes.labels.log-index']
                - equals:
                    kubernetes.container.name: "filebeat"
            config:
              - type: docker
                containers.ids:
                  - "${data.kubernetes.container.id}"
                exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
                fields:
                  log_topic: 'labels-${data.kubernetes.labels.log-index}'
            when:                       
              and:
                - has_fields: ['kubernetes.labels.log-index']
                - equals:
                    kubernetes.container.name: "filebeat"          
            config:
              - type: docker
                containers.ids:
                  - "${data.kubernetes.container.id}"
                exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
                fields:
                  log_topic: 'aip-${data.kubernetes.namespace}-${data.kubernetes.pod.name}'

Test2
use condition

Such as

  filebeat.autodiscover:
    providers:
      - type: kubernetes
        hints.enabled: true
        templates:
        - condition:
            and:
              has_fields: ['kubernetes.labels.log-index']
              not:
                equals:
                  kubernetes.container.name: "filebeat"
          config:
            - type: docker
              containers.ids:
                - "${data.kubernetes.container.id}"
              exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
                fields:
                  log_topic: 'labels-${data.kubernetes.labels.log-index}'
        - condition:                       
            and:
              not:
                has_fields: ['kubernetes.labels.log-index']
              not:
                equals:
                  kubernetes.container.name: "filebeat"         
          config:
            - type: docker
              containers.ids:
                - "${data.kubernetes.container.id}"
              exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
              fields:
                log_topic: 'aip-${data.kubernetes.namespace}-${data.kubernetes.pod.name}'

Test 3
Multiple type: kubernetes

Such as

 filebeat.autodiscover:
    providers:
      - type: kubernetes
        hints.enabled: true
        templates:
        - condition.and:
            - has_fields: ['kubernetes.labels.log-index']
            - not.equals.kubernetes.container.name: "filebeat"                 
          config:
            - type: docker
              containers.ids:
                - "${data.kubernetes.container.id}"
              exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
                fields:
                  log_topic: 'labels-${data.kubernetes.labels.log-index}'
      - type: kubernetes
        hints.enabled: true
        templates:
        - condition.and:                  
            - not.has_fields: ['kubernetes.labels.log-index']
            - not.equals.kubernetes.container.name: "filebeat"          
          config:
            - type: docker
              containers.ids:
                - "${data.kubernetes.container.id}"
              exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
              fields:
                log_topic: 'aip-${data.kubernetes.namespace}-${data.kubernetes.pod.name}'

But can't succeed, could you tell me whether there is something wrong with the grammar, thank you

2

Hello @icoolchn Could you edit your post and use the ``` or Preformatted test with your YAML configuration, badly indented YAML can make Filebeat not correctly applies your configuration.

Since you just copied the test without preformatting I cannot very if you have an error in your indentation.

4

Please help to see if the problem, thank you

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.