Filebeat multiline filter with autodiscover

Filebeat version: 7.5.2

I am trying to get my application running on Kubernetes with the ELK stack to do logging. The application is written in Java so I need to be able to able to ingest multiline stack traces as a single log message, I have a regex in Filebeat that does this. I want to apply this multiline filtering only to pods with the Kubernetes app label "my-app", the logs for everything else should be ingested regularly.

I am having a lot of trouble setting up autodiscovery, my config (from my k8s filebeat manifest) is below:

filebeat.yml: |-
    filebeat.autodiscover:
      providers:
          - type: kubernetes
            templates:
                - condition:
                    equals:
                        kubernetes.labels.app: "my-app"
                  config:
                      - type: container
                        #containers.ids:
                        #  - "${data.docker.container.id}"
                        #    -  "${data.kubernetes.container.id}" #"*"
                        paths:
                          - /var/log/container/*-${data.kubernetes.container.id}.log
                        multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
                        multiline.negate: true
              filebeat.config:
     #inputs: # prospectors formerly
        # Mounted `filebeat-prospectors` configmap, keep it in the prospector dir for now TODO
        #path: ${path.config}/prospectors.d/*.yml
        # Reload prospectors configs as they change:
        #reload.enabled: false
      modules:
        path: ${path.config}/modules.d/*.yml
        # Reload module configs as they change:
        # all modules are disabled currently
        reload.enabled: false
    output.logstash:
      hosts: ['logstash-service:5044']
          multiline.match: after

I tried adding a paths field and I tried using containers.ids. I still can't see my logs in Kibana using the above config. When using "containers.ids" Filebeat was complaining that a "paths" needs to be set, I set it and it still doesn't work. My cluster is working correctly since using the config below works but ignores other non-"my-app" logs that I want:

kubernetes.yml: |-
    - type: docker
      containers.ids:
      - "*"
      multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
      multiline.negate: true
      multiline.match: after
      processors:
        - add_kubernetes_metadata:
            in_cluster: true

If autodiscovery is not the way to do this then what is? I want a multiline filter applied to my Spring app logs (so stack traces get grouped as one message) and I want default (no filtering) applied to all other logs. Potentially I may want to add more custom rules in the future too. I have like 15 or so services running already ideally I don't want to add a rule for all of them, only the ones for which I don't want the default behavior for logging.

I got it to work using hints based autodiscover. If someone has a similar issue look at the docs here:
https://www.elastic.co/guide/en/beats/filebeat/master/configuration-autodiscover-hints.html

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.