# Where is add\_kubernetes\_metadata documented? **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [June 8, 2018, 8:14am UTC](https://discuss.elastic.co/t/where-is-add-kubernetes-metadata-documented/135110 "2018-06-08T08:14:47Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![TimWard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timward/32/19574_2.png) [@TimWard](https://discuss.elastic.co/u/TimWard)\ **Post date:** [June 8, 2018, 8:14am UTC](https://discuss.elastic.co/t/where-is-add-kubernetes-metadata-documented/135110/1 "2018-06-08T08:14:47Z")
At [https://www.elastic.co/guide/en/beats/filebeat/current/add-kubernetes-metadata.html](https://www.elastic.co/guide/en/beats/filebeat/current/add-kubernetes-metadata.html) there is an example of "default indexers and matchers disabled and enables ones that the user is interested in" but no links that I can see to anything that defines how you configure the add\_kubernetes\_metadata processor. The use case is that different pods/containers are going to output logs in different formats. For which there are different grok parsers in Logstash. So I need to add a custom field in the K8s deployment of Filebeat, just as I do in the standalone deployment, to tell Logstash which format a particular log line is in. Which I guess I want to do by label, or something like that? And there's no point in ingesting logs from applications I don't understand and can't parse, so I'll want to process data only from known containers/pods/labels/whatever. How do I do this please? ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [June 8, 2018, 9:21am UTC](https://discuss.elastic.co/t/where-is-add-kubernetes-metadata-documented/135110/2 "2018-06-08T09:21:49Z")
Hi @TimWard, There is an ongoing effort to improve this documentation: [https://github.com/elastic/beats/issues/5566](https://github.com/elastic/beats/issues/5566) As you guessed, what you want is to include some labels or annotations along with the rest of metadata added by `add_kubernetes_metadata`. You can achieve that by using these settings: ```auto processors: - add_kubernetes_metadata: include_annotations: - annotation_to_include ``` All labels are included by default ---
**Author:** ![TimWard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timward/32/19574_2.png) [@TimWard](https://discuss.elastic.co/u/TimWard)\ **Post date:** [June 8, 2018, 9:37am UTC](https://discuss.elastic.co/t/where-is-add-kubernetes-metadata-documented/135110/3 "2018-06-08T09:37:07Z")
If all labels are included by default that sounds like a good start. The questions are then: (1) How do I tell Filebeat not to process logs from pods that don't have labels I'm interested in? Eg "if there is no label called log\_type do not process events from this pod". (2) How do I set a custom field in the event depending on one of the labels? Eg "if there is a label called log\_type in this pod, add a custom field log\_type to the events from this pod". The point of (2) being that Logstash looks for particular values of a custom field in order to decide which grok parser to use and which Elasticsearch index to send the output to. Yes I have come to realise that I could do all this in Logstash by looking for the label metadata, but there are two problems with that approach: (a) vast amounts of resource could be wasted prospecting pods I'm not interested in, shipping their events to Logstash, and then throwing them away. (b) The Logstash code would then need to know about K8s deployment, and I think it would be cleaner if it didn't need different code depending on whether a particular application was or wasn't deployed in K8s. ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [June 8, 2018, 10:36am UTC](https://discuss.elastic.co/t/where-is-add-kubernetes-metadata-documented/135110/4 "2018-06-08T10:36:46Z")
Thank you for the explanation, it helps when we can understand your exact use case, it also helps to shape our roadmap 🙂 1. You can use autodiscover for this, check: [https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html) For instance: ```auto filebeat.autodiscover: providers: - kubernetes templates: - condition: regexp: kubernetes.labels.yourlabel: '.*' config: - type: docker containers.ids: - "${data.kubernetes.container.id}" # Point 2, add custom fields to events: fields: yourdesiredfield: "${data.kubernentes.labels.yourlabel}" ``` Also, 6.3 will bring a nice feature, allow you to define this behavior through kubernetes annotations, stay tunned for its release 🙂[https://github.com/elastic/beats/pull/7228](https://github.com/elastic/beats/pull/7228) Best regards ---
**Author:** ![TimWard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timward/32/19574_2.png) [@TimWard](https://discuss.elastic.co/u/TimWard)\ **Post date:** [June 8, 2018, 10:50am UTC](https://discuss.elastic.co/t/where-is-add-kubernetes-metadata-documented/135110/5 "2018-06-08T10:50:21Z")
Thanks very much - so many moving parts, so many new features 😀👍 (I'll close this when I get it working, which may or may not be today.) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2018, 10:50am UTC](https://discuss.elastic.co/t/where-is-add-kubernetes-metadata-documented/135110/6 "2018-07-06T10:50:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.