Filebeat not picking up /var/log/containers/*.log k8s

Elastic Stack Beats
1

I have filebeats deployed onto my kube cluster through the elastic helm chart.

The pod is privileged. Filebeats is picking up the new log that I create on the node. touch /var/log/containers/mylog.log and then I add stuff to it, and I see it in kibana. Which means that the filebeats -> kafka -> logstash -> elasticsearch -> kibana works. But I cannot pick up the other logs in that dir.

The logs in /var/log/containers/ are all symlinks.

Here is my config:

    filebeat.yml: |
    filebeat.inputs:
      - type: container
        enabled: true
        symlinks: true
        paths:
          - /var/log/containers/*.log
        processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"
      - type: log
        enabled: true
        symlinks: true
        paths:
          - /var/log/containers/*.log
        processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"
    output.kafka:
       enabled: true
       hosts: ["kafka.logging.svc.cluster.local:9092"]
       topic: "filebeat"
       codec.json:
          pretty: false
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.