K8S Node level logging as non-root user?

Elastic Stack Beats
1

I'm trying to perform node level logging in a kubernetes cluster using Filebeat. I've installed Filebeat as a daemonset and it is reading logs from /var/lib/docker/containers//*.log successfully. My concern is that I am having to run the Filebeat container as root because root owns /var/lib/docker/containers and the permissions for the directory are set to 700. I would like to avoid running containers as root for security concerns. I've looked at modifying the /var/lib/docker/containers permissions using an initcontainer, but that does not address any pods mounted after the Filebeat initcontainer runs. I looked at reading the logs through /var/log/pods/... but that would require using symbolic links and your docs recommend against relying on symbolic links.

What is considered best practice in a production environment:

  • avoiding node level logging altogether and just relying on sidecars for logging
  • running the filebeat container in a daemonset as root as I am doing
  • relying on symbolic links and managing those challenges

Or, is there an alternate approach that I am overlooking?

Thanks.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

4

Did you decide what to do here? - I've got a similar question to answer.

5

Hi!

Docker write container logs in the host with root permissions, hence you need to run Filebeat with privileged permissions to access them. So the suggested option should be the second one. Of course others might work too depending on the circumstances (for instance if you don't want autodiscover feature but just only capturing the logs of a specific app then sidecars could work too).

C.

6

@ChrsMark: Thanks for the reply. Since we do need to run autodiscover, we are going to continue with the second option: running the filebeat container in a daemonset as root.