Collect logs for specific containers or namespace in Openshift/Kubernetes

Elastic Stack Beats
2

Hey @sayeedc,

Yes, you can use hints-based autodiscover to provide a different configuration per pod or namespace. Hints are defined as annotations in your kubernetes resources.

In the main configuration file you would need this setup:

filebeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true
      add_resource_metadata.namespace.enabled: true
      hints.default_config.enabled: false

This tells the autodiscover provider to enable hints, to take into account namespace metadata, and as default configuration for each pod, it disables the inputs.

Then you can add configurations per namespace, or per pod. Adding an annotation like the following one will enable the inputs selectively:

annotations:
  co.elastic.logs/enabled: 'true'

If you do it in a pod, its logs will be collected. If you do it in a namespace, the logs of all the pods in this namespace will be collected.

You can see in the documentation other settings you can use per pod or namespace, apart of co.elastic.logs/enabled: https://www.elastic.co/guide/en/beats/filebeat/7.7/configuration-autodiscover-hints.html

1 Like