Filebeat on Kubernetes - How to get relevant kuberntes information into logs?

[Justin_Seiser]

Im using filebeat, installed via helm, from the official helm chart. We are sending logs to ElasticCloud.

My filebeat.yaml

setup.dashboards.enabled: true
logging.metrics:
enabled: false
filebeat.inputs:

• type: container
  paths:
  • /var/lib/docker/containers//.log
    processors:
  • add_kubernetes_metadata:
    in_cluster: true
  • add_cloud_metadata:
  • add_host_metadata:
    cloud.id: "xxx:xxx"
    cloud.auth: "elastic:xxxx"
    output.elasticsearch:
    hosts: "https://xxx.us-east-1.aws.found.io:9243"
    setup.kibana:
    host: "https://xxx.us-east-1.aws.found.io:9243"

Im getting logs, but im not getting things like.. the pod name, the namespace, any annotations or labels. So I cant really tell what log goes with what service.

Thanks,

[ChrsMark (Chris Mark)]

Hi @Justin_Seiser!

This can be achieved with add_kubernetes_metadata processor.

It seems that the Helm chart is a little bit outdated and in_cluster config option should not be there. I will send a fix for that however you could edit it manually if you don't want to wait for the upstream change. (patch)

Meanwhile could you check in Filebeat's logs for stuff related to add_kubernetes_metadata so as to make sure that the processor is being initialized without errors?

Thanks!