Filebeat autodiscover not working with hints & namespace

Elastic Stack Beats
1

I'd like to configure filebeat to use autodiscover with hints enabled.
Problem is, when I set a config for a certain namespace, it cancels the annotations in the pods under that namespace (meaning I get the multiline log split to different logs).

In filebeat's daemonset I configured it to be:

    filebeat.autodiscover:
          providers:
            - type: kubernetes
              node: ${NODE_NAME}
              hints.enabled: true
              hints.default_config:
                type: container
                paths:
                  - /var/log/containers/*-${data.kubernetes.container.id}.log
              include_annotations: '*'
              templates:
                - condition:
                    contains:
                      kubernetes.namespace: 'my-namespace'
                  config:
                    - type: container
                      paths:
                        - /var/log/containers/*-${data.kubernetes.container.id}.log
                      fields:
                        try: success

And in a pod that runs under the namespace 'my-namespace' I added those annotations:

      annotations:
        co.elastic.logs/multiline.type: 'pattern'
        co.elastic.logs/multiline.pattern: '^\[[0-9]{4}-[0-9]{2}-[0-9]{2}'
        co.elastic.logs/multiline.negate: 'true'
        co.elastic.logs/multiline.match: 'after'
        co.elastic.logs/exclude_lines: '^\n*$'

Any ideas on how to configure it so that i'll have a default configuration for that namespace but be able to use the annotations & hints?

2

If you have found the solution (even if it's in the docs), please post it, so if anybody else runs into it, they'll find the answer right away :slight_smile:

1 Like
3

The solution for my problem was to use appenders instead of template: https://www.elastic.co/guide/en/beats/filebeat/master/configuration-autodiscover-advanced.html
Appenders adds further config, where template overrides it.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.