# Kubernetes - ConfigMap **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 13, 2018, 9:18pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812 "2018-03-13T21:18:03Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Post date:** [March 13, 2018, 9:18pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/1 "2018-03-13T21:18:03Z")
Hi! I'm having an issue with Filebeat running in Kubernetes. I'm using the filebeat-kubernetes.yml file from the documentation and have made following change to the filebeat-prospectors ConfigMap: ``` > data: > kubernetes.yml: |- > - type: docker > containers.ids: > - "*" > processors: > - add_kubernetes_metadata: > in_cluster: true > filebeat.autodiscover: > providers: > - type: kubernetes > templates: > - condition: > equals: > kubernetes.namespace: development > config: > - type: docker > containers.ids: > - "${data.kubernetes.container.id}" ``` The pods are erroring with this message: > Unable to unpack config file due to error: missing field accessing '0.filebeat' (source:'/usr/share/filebeat/prospectors.d/kubernetes.yml') Any help as to where I went wrong would be greatly appreciated 🙂 Thanks! -Matt ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [March 13, 2018, 10:17pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/2 "2018-03-13T22:17:25Z")
Hi @bitva77, You are mixing a prospector configuration with autodiscover (which is a top level thing). If you plan to use autodiscover only you can drop `prospectors.yml` and put autodiscover settings in the main `filebeat.yml`. I created a sample gist doing so (disclaimer: I didn't test it): > Best regards ---
**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Post date:** [March 14, 2018, 12:16am UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/3 "2018-03-14T00:16:03Z")
OH! Thanks! I apparently have a hard time understanding things 🙂 lol Took a bit of massaging but here's a working config if anyone else is googling like I did: ``` --- apiVersion: v1 kind: ConfigMap metadata: name: filebeat-config namespace: kube-system labels: k8s-app: filebeat kubernetes.io/cluster-service: "true" data: filebeat.yml: |- filebeat.autodiscover: providers: - type: kubernetes templates: - condition: equals: kubernetes.namespace: development config: - type: docker containers.ids: - "${data.kubernetes.container.id}" ``` Let me ask you, when running in K8s, why wouldn't you use autodiscover? Thanks again! ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [March 14, 2018, 12:33am UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/4 "2018-03-14T00:33:21Z")
We released it pretty recently, at some point it should be the default in the manifests, once we consider it stable enough. Your feedback is important here, too, we learn from what users report and request 🙂 ---
**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Post date:** [March 14, 2018, 5:14pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/5 "2018-03-14T17:14:00Z")
cool beans. I'll just keep putting updates and stuff here then. I have a question below: Our goal is to capture logs from our microservices and not all the crap that comes from K8s, so I've limited filebeats to the namespaces we care about and so far so good with that. We also send everything to Logstash instead. Here's what I got so far for the filebeat-config ConfigMap: ``` --- apiVersion: v1 kind: ConfigMap metadata: name: filebeat-config namespace: kube-system labels: k8s-app: filebeat kubernetes.io/cluster-service: "true" data: filebeat.yml: |- filebeat.autodiscover: providers: - type: kubernetes templates: - condition: equals: kubernetes.namespace: development - condition: equals: kubernetes.namespace: sqa - condition: equals: kubernetes.namespace: stage config: - type: docker containers.ids: - "${data.kubernetes.container.id}" output.logstash: hosts: ['${LOGSTASH_HOST}:${LOGSTASH_PORT}'] ``` I pass the actual LOGSTASH\_HOST and \_PORT later on in the DaemonSet configuration. I commented out the rest of the prospector references in the original YAML as well since we aren't using any of that. Question: Is there a need for me to specify filebeat.registry\_file ? and have that be a physical location on the host (via hostPath)? If something happens to the filebeat pod and it gets restarted, it's going to resend all the same logs from the namespaces I've identified. To avoid that, I need that registry file to persist, correct? ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [March 14, 2018, 5:30pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/6 "2018-03-14T17:30:11Z")
You don't need to update `registry_file` setting, as it defaults to the `data` volume. It's recommended to mount the `data` volume as a hostPath, so we persist the resgistry file across pod changes. That will avoid us to send logs again on restart. I recently added a warning about that here: [https://github.com/elastic/beats/pull/6526](https://github.com/elastic/beats/pull/6526) ---
**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Post date:** [March 14, 2018, 7:35pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/7 "2018-03-14T19:35:19Z")
Gotcha. FYI for anyone googlin, to persist the `data` volume, change the data volume definition from `emptyDir` to this: ``` - name: data hostPath: path: /var/lib/filebeat ``` of course you need the directory /var/lib/filebeat to exist on every node in your cluster and then the registry file should persist no matter what happens to the pods. ---
**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Post date:** [March 15, 2018, 8:47pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/8 "2018-03-15T20:47:21Z")
A few things now: 1. It appears only the last condition statement works. Meaning in the above configuration, only the "stage" namespace is being looked at. 2. Our logs are sent by the apps are multi-line but are not being sent by Filebeat via multi-line no matter what I try to make the multiline.pattern. Is that not observed with autodiscover? Thanks again! ---
**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Post date:** [March 16, 2018, 3:06am UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/9 "2018-03-16T03:06:43Z")
Alrighty, I figured out the multi-conditional statements to only look at certain namespaces with this configuration: ``` data: filebeat.yml: |- filebeat.autodiscover: providers: - type: kubernetes in_cluster: true templates: - condition: or: - equals: kubernetes.namespace: development - equals: kubernetes.namespace: sqa - equals: kubernetes.namespace: stage config: - type: docker containers.ids: - "${data.kubernetes.container.id}" ``` I'm still having issues with getting multiline.pattern to work though...will keep at it until I pass out here. Any advice you can give me on that (when I awake from my coma) would be appreciated! Here's a sample of the logs that our application is sending: > {"log":"\u001b[40m\u001b[32minfo\u001b[39m\u001b[22m\u001b[49m: Microsoft.AspNetCore.Mvc.Internal.ObjectResultExecutor[1]\n","stream":"stdout","time":"2018-03-15T17:58:43.269647888Z"} > {"log":" Executing ObjectResult, writing value Microsoft.AspNetCore.Mvc.ControllerContext.\n","stream":"stdout","time":"2018-03-15T17:58:43.269673063Z"} I have this under providers: > ``` > multiline.pattern: '^\\u001b' > multiline.negate: true > multiline.match: after > > ``` Thanks! ---
**Author:** ![bitva77](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bitva77/32/66694_2.png) [@bitva77](https://discuss.elastic.co/u/bitva77)\ **Post date:** [March 16, 2018, 4:35pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/10 "2018-03-16T16:35:04Z")
I think I got it. All in the indentation: ``` data: filebeat.yml: |- filebeat.autodiscover: providers: - type: kubernetes in_cluster: true templates: - condition: or: - equals: kubernetes.namespace: development - equals: kubernetes.namespace: sqa - equals: kubernetes.namespace: stage config: - type: docker containers.ids: - "${data.kubernetes.container.id}" multiline.pattern: '^[[:space:]]' multiline.negate: false multiline.match: after output.logstash: hosts: ['${LOGSTASH_HOST}:${LOGSTASH_PORT}'] ``` sorry for all the back and forth with myself here. Just trying to document the steps as this autodiscover stuff seems pretty new. Thanks! -Matt ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 13, 2018, 4:35pm UTC](https://discuss.elastic.co/t/kubernetes-configmap/123812/11 "2018-04-13T16:35:09Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.