Kubernetes filebeat best practices

[lbchv (Andrew)]

The documentation says to avoid harvesting symlinks when dialing with file rotation (link):

When dealing with file rotation, avoid harvesting symlinks. Instead use the paths setting to point to the original file, and specify a pattern that matches the file you want to harvest and all of its rotated files.

But at the same time in the section about kubernetes autodiscover symlink example given in path (link):

filebeat.autodiscover:
  providers:
    - type: kubernetes
      templates:
        - condition:
            equals:
              kubernetes.namespace: kube-system
          config:
            - type: container
              paths:
                - /var/log/containers/*-${data.kubernetes.container.id}.log
              exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines

Symlinks in /var/log/containers*.log points to log files in /var/log/pods which are rotated by kubelet.

What is best practices? Is there documentation mistake?

[khoa102 (Khoa)]

Hi, I am also interested in this topic. Did you find something about this? I use this setting for my Kubernetes cluster and it worked well before. But recently, there seems to be a problem with either log rotation or heavy load. I would like to know the best practices for setting up Filebeat on Kubernetes to see if this is related to the problem I encounter.