Filebeat how to autodiscover kubernetes service and fetch pod log?

I've deploy filebeat on kubernetes cluster following the doc. And set resource: service in the config file:

    filebeat.autodiscover:
      providers:
        - type: kubernetes
          resource: service
          node: ${NODE_NAME}
          include_annotations: []
          include_labels: []
          hints.enabled: true
          hints.default_config:
            enabled: false
            type: container
            paths:
              - /var/log/containers/*-${data.kubernetes.container.id}.log
    output.console:
      pretty: true

And then I add co.elastic.logs/enabled: "true" on a Service.

Filebeats discovered that service but print a error:

2020-06-22T03:27:25.307Z ERROR [autodiscover] autodiscover/autodiscover.go:210 Auto discover config check failed for config '{
"docker-json": {
"cri_flags": true,
"format": "auto",
"partial": true,
"stream": "all"
},
"symlinks": true,
"type": "container"
}', won't start runner: each input must have at least one path defined

So, did I missing something?

Solved. I use config template solve this.

Hi Daniel,
I would be very interested if you could post the solution, I've got the same pb
Regards
R.

Something like this, but you cannot specify the log file name, because kubernetes log file name is {pod.name}-{pod.namespace}-{container.name}-{container.id}.log

filebeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            equals:
              kubernetes.container.image: "redis"
          config:
            - module: redis
              log:
                input:
                  type: container
                  paths:
                    - /var/log/containers/*-${data.kubernetes.container.id}.log

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.