Filebeat on Kubernetes not collecting logs from master nodes

colincoghill · August 28, 2019, 9:27pm

Hi,

While trying to investigate a problem with our kubernetes cluster I discovered that our filebeat install was not collecting logs from anything running on the "master" nodes. It did when I initially installed filebeat a while ago.

After some digging, I find that a few kubernetes versions ago, they changed it so that DaemonSets (which filebeat uses to deploy on kubernetes) do not deploy to master nodes by default.

I believe the filebeat-kubernetes file at :

github.com

elastic/beats/blob/master/deploy/kubernetes/filebeat/filebeat-daemonset.yaml

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: kube-system
  labels:
    k8s-app: filebeat
spec:
  template:
    metadata:
      labels:
        k8s-app: filebeat
    spec:
      serviceAccountName: filebeat
      terminationGracePeriodSeconds: 30
      hostNetwork: true
      dnsPolicy: ClusterFirstWithHostNet
      containers:
      - name: filebeat
        image: docker.elastic.co/beats/filebeat:%VERSION%

This file has been truncated. show original

Needs to be changed to include

    spec:
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule

This works for me.

Am I correct? is this something I should submit as a bug?

links:
https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/

jsoriano · August 29, 2019, 8:56pm

Hi @colincoghill and welcome

Thanks for reporting this, I think we should add these tolerations to the reference configuration and/or to the documentation. I see we already mention something similar for OpenShift.

Please, submit a bug for this.

system · September 26, 2019, 8:56pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.