# Filebeat on kubernetes - access denied

**URL:** <https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [October 16, 2018, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678 "2018-10-16T14:00:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\
**Post date:** [October 16, 2018, 2:00pm UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/1 "2018-10-16T14:00:40Z")

</div>

Ive created a filebeat daemonset for my k8 cluster using (roughly) these params: [https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat/filebeat-daemonset.yaml](https://github.com/elastic/beats/blob/master/deploy/kubernetes/filebeat/filebeat-daemonset.yaml).

Of 5 cluster hosts 4 are working fine and Im getting data. On the 5th Im seeing this error message:

```
2018-10-16T13:53:55.243Z	INFO	instance/beat.go:544	Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2018-10-16T13:53:55.244Z	INFO	instance/beat.go:315	filebeat stopped.
2018-10-16T13:53:55.244Z	ERROR	instance/beat.go:743	Exiting: Failed to create Beat meta file: open /usr/share/filebeat/data/meta.json.new: permission denied
Exiting: Failed to create Beat meta file: open /usr/share/filebeat/data/meta.json.new: permission denied

```

The 5 hosts should be identical. (They were provisioned via puppet).

When I look for that path (/usr/share/filebeat/data) I don't see it on _any_ of the 5 hosts. For some reason filebeat doesn't seem to complain on 4/5 though.

Any ideas?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 16, 2018, 9:33pm UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/2 "2018-10-16T21:33:52Z")

</div>

Filebeat needs to be the owner of `/usr/share/filebeat`. Are you sure the directory does not exist? The logs indicate filebeat even sets the home path to `/usr/share/filebeat`.

How does your setup differ from the docker images + daemonset in the beats repo?

---

<div class="post-metadata">

**Author:** ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\
**Post date:** [October 17, 2018, 10:23am UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/3 "2018-10-17T10:23:18Z")

</div>

/usr/share/filebeat is root:root on all 5 machines. The 'data' folder doesn't exist on any of them.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [October 17, 2018, 11:44am UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/4 "2018-10-17T11:44:07Z")

</div>

How does your setup differ from the docker images + daemonset in the beats repo?

---

<div class="post-metadata">

**Author:** ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\
**Post date:** [October 17, 2018, 10:50pm UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/5 "2018-10-17T22:50:55Z")

</div>

I pointed it at a logstash host. I can't think of anything else major (or minor for that matter).

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [October 19, 2018, 8:50am UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/6 "2018-10-19T08:50:06Z")

</div>

Hi @ethrbunny, could you please report the manifest you are using and Kubernetes version?

Our default manifest is supposed to create the data folder for you, with the correct permissions.

Best regards

---

<div class="post-metadata">

**Author:** ![ethrbunny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ethrbunny/32/34603_2.png) [@ethrbunny](https://discuss.elastic.co/u/ethrbunny)\
**Post date:** [October 19, 2018, 9:57am UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/7 "2018-10-19T09:57:40Z")

</div>

Kubernetes 1.12

Here's the manifest:

```
  ---
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: filebeat-config
    namespace: kube-system
    labels:
      k8s-app: filebeat
  data:
    filebeat.yml: |-
      filebeat.config:
        inputs:
          # Mounted `filebeat-inputs` configmap:
          path: ${path.config}/inputs.d/*.yml
          # Reload inputs configs as they change:
          reload.enabled: false
        modules:
          path: ${path.config}/modules.d/*.yml
          # Reload module configs as they change:
          reload.enabled: false

      # To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
      filebeat.autodiscover:
        providers:
          - type: kubernetes
            hints.enabled: true
        templates:
        # - condition:
        # regexp:
        # kubernetes.labels.yourlabel: '.*'
            config:
              - type: docker
                containers.ids:
                  - "${data.kubernetes.container.id}"
                # Point 2, add custom fields to events:
                fields:
                  yourdesiredfield: "${data.kubernentes.labels.yourlabel}"
                
      processors:
        - add_cloud_metadata:

      output.logstash:
        hosts: ["10.95.96.75:5044"]
  ---
  apiVersion: v1
  kind: ConfigMap
  metadata:
    name: filebeat-inputs
    namespace: kube-system
    labels:
      k8s-app: filebeat
  data:
    kubernetes.yml: |-
      - type: docker
        containers.ids:
        - "*"
        processors:
          - add_kubernetes_metadata:
              in_cluster: true
  ---
  apiVersion: extensions/v1beta1
  kind: DaemonSet
  metadata:
    name: filebeat
    namespace: kube-system
    labels:
      k8s-app: filebeat
  spec:
    template:
      metadata:
        labels:
          k8s-app: filebeat
      spec:
        serviceAccountName: filebeat
        terminationGracePeriodSeconds: 30
        containers:
        - name: filebeat
          image: docker.elastic.co/beats/filebeat:6.4.1
          args: [
            "-c", "/etc/filebeat.yml",
            "-e",
          ]
          env:
    # - name: ELASTICSEARCH_HOST
    # value: elasticsearch
    # - name: ELASTICSEARCH_PORT
    # value: "9200"
    # - name: ELASTICSEARCH_USERNAME
    # value: elastic
    # - name: ELASTICSEARCH_PASSWORD
    # value: changeme
    # - name: ELASTIC_CLOUD_ID
    # value:
    # - name: ELASTIC_CLOUD_AUTH
    # value:
          securityContext:
            runAsUser: 0
          resources:
            limits:
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
          - name: config
            mountPath: /etc/filebeat.yml
            readOnly: true
            subPath: filebeat.yml
          - name: inputs
            mountPath: /usr/share/filebeat/inputs.d
            readOnly: true
          - name: data
            mountPath: /usr/share/filebeat/data
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
            readOnly: true
          - name: msgs
            mountPath: /var/log/messages
            readOnly: true
        volumes:
        - name: config
          configMap:
            defaultMode: 0600
            name: filebeat-config
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: msgs
          hostPath:
            path: /var/log/messages
        - name: inputs
          configMap:
            defaultMode: 0600
            name: filebeat-inputs
        # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
        - name: data
          hostPath:
            path: /var/lib/filebeat-data
            type: DirectoryOrCreate
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    name: filebeat
  subjects:
  - kind: ServiceAccount
    name: filebeat
    namespace: kube-system
  roleRef:
    kind: ClusterRole
    name: filebeat
    apiGroup: rbac.authorization.k8s.io
  ---
  apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRole
  metadata:
    name: filebeat
    labels:
      k8s-app: filebeat
  rules:
  - apiGroups: [""] # "" indicates the core API group
    resources:
    - namespaces
    - pods
    verbs:
    - get
    - watch
    - list
  ---
  apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: filebeat
    namespace: kube-system
    labels:
      k8s-app: filebeat
  ---
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2018, 9:57am UTC](https://discuss.elastic.co/t/filebeat-on-kubernetes-access-denied/152678/8 "2018-11-16T09:57:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.