# Filebeat not starting as deamonset on Kuberneters

**URL:** <https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 15, 2018, 1:27pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992 "2018-02-15T13:27:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![irina\_andronachi](https://avatars.discourse-cdn.com/v4/letter/i/e19b73/32.png) [@irina\_andronachi](https://discuss.elastic.co/u/irina_andronachi)\
**Post date:** [February 15, 2018, 1:27pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992/1 "2018-02-15T13:27:41Z")

</div>

Hi,

I configured a deamonset on a kubernetes cluster

apiVersion: v1  
items:

- apiVersion: extensions/v1beta1  
kind: DaemonSet  
metadata:  
name: tracing-filebeat  
namespace: ""  
spec:  
template:  
metadata:  
labels:  
tracing: filebeat  
type: tracing  
spec:  
containers:  
- env:  
- name: LOGSTASH\_HOST  
value: logstash:5044  
image: docker-registry.default.svc:5000/filebeat:1.4  
name: tracing-filebeat  
volumeMounts:  
- mountPath: /var/log/containers  
name: varlog  
serviceAccountName: filebeat  
volumes:  
- hostPath:  
path: /var/log/containers/  
name: varlog

My docker file looks like this  
FROM [docker.elastic.co/beats/filebeat:6.2.0](http://docker.elastic.co/beats/filebeat:6.2.0)  
COPY filebeat.yml /usr/share/filebeat/filebeat.yml  
USER root  
RUN chown filebeat /usr/share/filebeat/filebeat.yml  
RUN chmod go-w /usr/share/filebeat/filebeat.yml  
USER filebeat

The pod fails with the error "/usr/local/bin/docker-entrypoint: line 8: exec: filebeat: not found"  
The issue seems to be related to the volume mount. If I remove the volumes the pods start immediately.  
Do you have any idea what could be wrong?

The issue seems to be related to any volume mounts, it doesn't matter the path or the content.

Thank you,  
Irina

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [February 15, 2018, 1:42pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992/2 "2018-02-15T13:42:21Z")

</div>

That looks like a very old image. Why are you not using `docker.elastic.co/beats/filebeat:6.2.1`, which I believe is the latest official one ?

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [February 15, 2018, 1:52pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992/3 "2018-02-15T13:52:56Z")

</div>

Hi @irina_andronachi,

We provide reference manifests to deploy Filebeat as a daemonset, probably you want to have a look to them. That will allow you to use ConfigMaps instead of creating a new image for Filebeat: [https://www.elastic.co/guide/en/beats/filebeat/6.2/running-on-kubernetes.html](https://www.elastic.co/guide/en/beats/filebeat/6.2/running-on-kubernetes.html)

---

<div class="post-metadata">

**Author:** ![irina\_andronachi](https://avatars.discourse-cdn.com/v4/letter/i/e19b73/32.png) [@irina\_andronachi](https://discuss.elastic.co/u/irina_andronachi)\
**Post date:** [February 15, 2018, 2:51pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992/4 "2018-02-15T14:51:42Z")

</div>

Hi,  
with the rerefence manifest I get exactly the same error  
"/usr/local/bin/docker-entrypoint: line 8: exec: filebeat: not found"  
I have only removed the "securityContext: runAsUser: 0" as it failed with an error stating that the user has to be in a range.

* * *

apiVersion: v1  
kind: ConfigMap  
metadata:  
name: filebeat-config  
namespace: edcm  
labels:  
k8s-app: filebeat  
[kubernetes.io/cluster-service:](http://kubernetes.io/cluster-service:) "true"  
data:  
filebeat.yml: |-  
filebeat.config:  
prospectors:  
# Mounted `filebeat-prospectors` configmap:  
path: ${path.config}/prospectors.d/_.yml  
# Reload prospectors configs as they change:  
reload.enabled: false  
modules:  
path: ${path.config}/modules.d/_.yml  
# Reload module configs as they change:  
reload.enabled: false

```
processors:
  - add_cloud_metadata:

cloud.id: ${ELASTIC_CLOUD_ID}
cloud.auth: ${ELASTIC_CLOUD_AUTH}

output.elasticsearch:
  hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
  username: ${ELASTICSEARCH_USERNAME}
  password: ${ELASTICSEARCH_PASSWORD}

```

* * *

apiVersion: v1  
kind: ConfigMap  
metadata:  
name: filebeat-prospectors  
namespace: edcm  
labels:  
k8s-app: filebeat  
[kubernetes.io/cluster-service:](http://kubernetes.io/cluster-service:) "true"  
data:  
kubernetes.yml: |-  
- type: docker  
containers.ids:  
- "\*"  
processors:  
- add\_kubernetes\_metadata:  
in\_cluster: true

* * *

apiVersion: extensions/v1beta1  
kind: DaemonSet  
metadata:  
name: filebeat  
namespace: edcm  
labels:  
k8s-app: filebeat  
[kubernetes.io/cluster-service:](http://kubernetes.io/cluster-service:) "true"  
spec:  
template:  
metadata:  
labels:  
k8s-app: filebeat  
[kubernetes.io/cluster-service:](http://kubernetes.io/cluster-service:) "true"  
spec:  
serviceAccountName: filebeat  
terminationGracePeriodSeconds: 30  
containers:  
- name: filebeat  
image: [docker.elastic.co/beats/filebeat:6.1.3](http://docker.elastic.co/beats/filebeat:6.1.3)  
args: [  
"-c", "/etc/filebeat.yml",  
"-e",  
]  
env:  
- name: ELASTICSEARCH\_HOST  
value: elasticsearch  
- name: ELASTICSEARCH\_PORT  
value: "9200"  
- name: ELASTICSEARCH\_USERNAME  
value: elastic  
- name: ELASTICSEARCH\_PASSWORD  
value: changeme  
- name: ELASTIC\_CLOUD\_ID  
value:  
- name: ELASTIC\_CLOUD\_AUTH  
value:  
resources:  
limits:  
memory: 200Mi  
requests:  
cpu: 100m  
memory: 100Mi  
volumeMounts:  
- name: config  
mountPath: /etc/filebeat.yml  
readOnly: true  
subPath: filebeat.yml  
- name: prospectors  
mountPath: /usr/share/filebeat/prospectors.d  
readOnly: true  
- name: data  
mountPath: /usr/share/filebeat/data  
- name: varlibdockercontainers  
mountPath: /var/lib/docker/containers  
readOnly: true  
volumes:  
- name: config  
configMap:  
defaultMode: 0600  
name: filebeat-config  
- name: varlibdockercontainers  
hostPath:  
path: /var/lib/docker/containers  
- name: prospectors  
configMap:  
defaultMode: 0600  
name: filebeat-prospectors  
- name: data  
emptyDir: {}

* * *

apiVersion: [rbac.authorization.k8s.io/v1beta1](http://rbac.authorization.k8s.io/v1beta1)  
kind: ClusterRoleBinding  
metadata:  
name: filebeat  
subjects:

- kind: ServiceAccount  
name: filebeat  
namespace: edcm  
roleRef:  
kind: ClusterRole  
name: filebeat  
apiGroup: [rbac.authorization.k8s.io](http://rbac.authorization.k8s.io)

* * *

apiVersion: [rbac.authorization.k8s.io/v1beta1](http://rbac.authorization.k8s.io/v1beta1)  
kind: ClusterRole  
metadata:  
name: filebeat  
labels:  
k8s-app: filebeat  
rules:

- apiGroups: [""] # "" indicates the core API group  
resources:
  - namespaces
  - pods  
verbs:
  - get
  - watch
  - list

* * *

Thank you,  
Irina

---

<div class="post-metadata">

**Author:** ![irina\_andronachi](https://avatars.discourse-cdn.com/v4/letter/i/e19b73/32.png) [@irina\_andronachi](https://discuss.elastic.co/u/irina_andronachi)\
**Post date:** [February 15, 2018, 2:54pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992/5 "2018-02-15T14:54:02Z")

</div>

Here is how the pod looks like  
[ec2-user@ip-10-0-117-21 ~]$ oc describe pod filebeat-9s79q  
Name: filebeat-9s79q  
Namespace: edcm  
Node: ip-10-0-117-24.eu-west-1.compute.internal/10.0.117.24  
Start Time: Thu, 15 Feb 2018 15:39:36 +0100  
Labels: controller-revision-hash=4091261649  
k8s-app=filebeat  
[kubernetes.io/cluster-service=true](http://kubernetes.io/cluster-service=true)  
pod-template-generation=1  
Annotations: [kubernetes.io/created-by=](http://kubernetes.io/created-by=){"kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"DaemonSet","namespace":"edcm","name":"filebeat","uid":"0b86758a-125e-11e8-a3a9-0a4d7e5abdea","apiVersion":...  
[openshift.io/scc=hostaccess](http://openshift.io/scc=hostaccess)  
Status: Running  
IP: 10.130.0.8  
Created By: DaemonSet/filebeat  
Controlled By: DaemonSet/filebeat  
Containers:  
filebeat:  
Container ID: docker://5b1865edfc72a6571196246296131d366231ccfdf4a8522edb9cba4c3ece1f4f  
Image: [docker.elastic.co/beats/filebeat:6.1.3](http://docker.elastic.co/beats/filebeat:6.1.3)  
Image ID: docker-pullable://docker.elastic.co/beats/filebeat@sha256:ffb7a446d6f4931f5458045acded76d4a3a7a08f69832ecf1f496a3fcd4e39a4  
Port:   
Args:  
-c  
/etc/filebeat.yml  
-e  
State: Waiting  
Reason: CrashLoopBackOff  
Last State: Terminated  
Reason: Error  
Exit Code: 127  
Started: Thu, 15 Feb 2018 15:45:31 +0100  
Finished: Thu, 15 Feb 2018 15:45:31 +0100  
Ready: False  
Restart Count: 6  
Limits:  
memory: 200Mi  
Requests:  
cpu: 100m  
memory: 100Mi  
Environment:  
ELASTICSEARCH\_HOST: elasticsearch  
ELASTICSEARCH\_PORT: 9200  
ELASTICSEARCH\_USERNAME: elastic  
ELASTICSEARCH\_PASSWORD: changeme  
ELASTIC\_CLOUD\_ID:   
ELASTIC\_CLOUD\_AUTH:   
Mounts:  
/etc/filebeat.yml from config (ro)  
/usr/share/filebeat/data from data (rw)  
/usr/share/filebeat/prospectors.d from prospectors (ro)  
/var/lib/docker/containers from varlibdockercontainers (ro)  
/var/run/secrets/kubernetes.io/serviceaccount from filebeat-token-mq4f2 (ro)  
Conditions:  
Type Status  
Initialized True  
Ready False  
PodScheduled True  
Volumes:  
config:  
Type: ConfigMap (a volume populated by a ConfigMap)  
Name: filebeat-config  
Optional: false  
varlibdockercontainers:  
Type: HostPath (bare host directory volume)  
Path: /var/lib/docker/containers  
prospectors:  
Type: ConfigMap (a volume populated by a ConfigMap)  
Name: filebeat-prospectors  
Optional: false  
data:  
Type: EmptyDir (a temporary directory that shares a pod's lifetime)  
Medium:   
filebeat-token-mq4f2:  
Type: Secret (a volume populated by a Secret)  
SecretName: filebeat-token-mq4f2  
Optional: false  
QoS Class: Burstable  
Node-Selectors:   
Tolerations: [node.alpha.kubernetes.io/notReady:NoExecute](http://node.alpha.kubernetes.io/notReady:NoExecute)  
[node.alpha.kubernetes.io/unreachable:NoExecute](http://node.alpha.kubernetes.io/unreachable:NoExecute)  
Events:  
FirstSeen LastSeen Count From SubObjectPath Type Reason Message

* * *

7m 7m 1 kubelet, ip-10-0-117-24.eu-west-1.compute.internal Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "data"  
7m 7m 1 kubelet, ip-10-0-117-24.eu-west-1.compute.internal Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "varlibdockercontainers"  
7m 7m 1 kubelet, ip-10-0-117-24.eu-west-1.compute.internal Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "prospectors"  
7m 7m 1 kubelet, ip-10-0-117-24.eu-west-1.compute.internal Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "config"  
7m 7m 1 kubelet, ip-10-0-117-24.eu-west-1.compute.internal Normal SuccessfulMountVolume MountVolume.SetUp succeeded for volume "filebeat-token-mq4f2"  
7m 7m 1 kubelet, ip-10-0-117-24.eu-west-1.compute.internal spec.containers{filebeat} Normal Pulling pulling image "[docker.elastic.co/beats/filebeat:6.1.3](http://docker.elastic.co/beats/filebeat:6.1.3)"  
7m 7m 1 kubelet, ip-10-0-117-24.eu-west-1.compute.internal spec.containers{filebeat} Normal Pulled Successfully pulled image "[docker.elastic.co/beats/filebeat:6.1.3](http://docker.elastic.co/beats/filebeat:6.1.3)"  
7m 6m 3 kubelet, ip-10-0-117-24.eu-west-1.compute.internal spec.containers{filebeat} Normal Pulled Container image "[docker.elastic.co/beats/filebeat:6.1.3](http://docker.elastic.co/beats/filebeat:6.1.3)" already present on machine  
7m 6m 4 kubelet, ip-10-0-117-24.eu-west-1.compute.internal spec.containers{filebeat} Normal Created Created container  
7m 6m 4 kubelet, ip-10-0-117-24.eu-west-1.compute.internal spec.containers{filebeat} Normal Started Started container  
7m 2m 23 kubelet, ip-10-0-117-24.eu-west-1.compute.internal spec.containers{filebeat} Warning BackOff Back-off restarting failed container  
[ec2-user@ip-10-0-117-21 ~]$ oc logs -f filebeat-7s9ct

---

<div class="post-metadata">

**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\
**Post date:** [February 19, 2018, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992/6 "2018-02-19T15:22:29Z")

</div>

Thank you for the details, it seems logs are missing from the paste, could you include them?

Best regards

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 19, 2018, 3:22pm UTC](https://discuss.elastic.co/t/filebeat-not-starting-as-deamonset-on-kuberneters/119992/7 "2018-03-19T15:22:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.