Filebeat k8s annotations

Elastic Stack Beats
1

I'm trying to get filebeat working in kubernetes, I'm wanting to only collect logs for pods in specific namespaces

here is my filebeat config

filebeat.inputs:
  - type: log
    paths:
      - /var/lib/rancher/rke/log/*.log
      - /var/log/syslog
    symlinks: true
  - type: log
    enabled: true
    paths:
      - /var/log/kube-audit/audit-log.json
filebeat.autodiscover:
  providers:
    - type: kubernetes
      node: ${NODE_NAME}
      hints.enabled: true
      hints.default_config.enabled: false
      add_resource_metadata:
        namespace:
          include_annotations: ["nsannotation1"]
processors:
  - timestamp:
      field: filebeat_timestamp
      layouts:
        - '2006-01-02T15:04:05Z'
  - add_fields:
      fields:
        cluster: ${CLUSTER}
        DC: ${DC}
        node: ${NODE_NAME}
  - dissect:
      when:
        contains:
          log.file.path: "/var/lib/rancher/rke/log/"
      tokenizer: "%{?key1}/%{?key}/%{?key}/%{?key}/%{?key}/%{?key}/%{container.name}_%{}"
      field: "log.file.path"
      target_prefix: ""
setup.dashboards.enabled: false
setup.template.enabled: false
setup.template.overwrite: false
logging.level: info
output.logstash:
  hosts: [logstash.netskope.io:5044]
  ssl:
    certificate_authorities: ["/etc/beats/certificates/ca.crt"]
    certificate: "/etc/beats/certificates/client.crt"
    key: "/etc/beats/certificates/client.key"

and here are the annotations on my namespace

apiVersion: v1
kind: Namespace
metadata:
  annotations:
    cattle.io/status: '{"Conditions":[{"Type":"ResourceQuotaInit","Status":"True","Message":"","LastUpdateTime":"2021-05-05T14:43:41Z"},{"Type":"InitialRolesPopulated","Status":"True","Message":"","LastUpdateTime":"2021-05-05T14:43:47Z"}]}'
    co.elastic.logs/enabled: "true"
    co.elastic.logs/fileset.stderr: error
    co.elastic.logs/fileset.stdout: access
    co.elastic.logs/module: nginx
    field.cattle.io/projectId: c-pgth7:p-8nnk2
    lifecycle.cattle.io/create.namespace-auth: "true"
  creationTimestamp: "2021-05-05T14:42:36Z"
  finalizers:
  - controller.cattle.io/namespace-auth
  labels:
    field.cattle.io/projectId: p-8nnk2
  name: default
  resourceVersion: "594217633"
  selfLink: /api/v1/namespaces/default
  uid: ea8bd3eb-7886-45b2-a1f1-2c3f8617cb3d
spec:
  finalizers:
  - kubernetes
status:
  phase: Active

any idea what I am doing wrong here?

2

Hi @Jeff_d_Ambly!

It seems that you are trying to use Hints based autodiscovery, note here: The hints system looks for hints in Kubernetes Pod annotations, it does not apply to the namespace.

For you case - to collect logs for pods in specific namespace only, I would suggest to use configuration similar to the one provided in this doc:

filebeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            equals:
              kubernetes.namespace: kube-system  # set desired namespace here 
          config:
            - type: container
              paths:
                - /var/log/containers/*.log

You don't need to add any annotations in this case.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.