# Filebeat deployment in Kubernetes/Docker **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [January 2, 2018, 10:53am UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759 "2018-01-02T10:53:17Z")\ **Posts on this page:** 8\ **Page:** 2
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [January 16, 2018, 11:18am UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/21 "2018-01-16T11:18:16Z")
Hi, as you probably know this is a quite new feature 🙂, we are continually adding new features to it to improve user experience, so I want to thank you for your feedback, it helps us to shape future features. For the case of NGINX, it sends access logs to `stdout`, and error logs to `stderr`. I recently [implemented a way to filter on the stream](https://github.com/elastic/beats/pull/6057), so you can pass the correct output to each fileset of a module. This is how it will look in filebeat: ```auto filebeat.autodiscover: providers: - type: docker templates: - condition: contains: docker.container.name: "nginx" config: - module: nginx access: prospector: type: docker containers.stream: stdout container.ids: - "${data.docker.container.id}" error: prospector: type: docker containers.stream: stderr container.ids: - "${data.docker.container.id}" ``` I've opened a [new issue to allow to define a default fallback](https://github.com/elastic/beats/issues/6084), in case none of the templates matches. As a note, you don't need to define `paths` parameter, it's already defined for you ---
**Author:** ![Asher\_Shoshan](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@Asher\_Shoshan](https://discuss.elastic.co/u/Asher_Shoshan)\ **Post date:** [January 16, 2018, 1:25pm UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/22 "2018-01-16T13:25:23Z")
Thanks, 1. It was not working otherwise... I have to put the path of the exact container (see my config) 2. Only 'access' file-set is working - if I put bot (like your config) - all messages end in error in elasicsearch (because actually there is only one log file in container of both streams) 3. kubernetes metadata fields ignored in the condition part (when launched by kubernetes). 4. What about my apps logs (not nginx, apache, etc) - how to define it?? ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [January 16, 2018, 3:33pm UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/23 "2018-01-16T15:33:01Z")
1. I see now why it wasn't working for you, there is a typo: `container.ids` -\> `containers.ids`. 2. It depends on how nginx is configured, it must output error log to stderr. Also if you see parsing errors, it's because `stream` filter is not yet working (unreleased). 3. you are using `docker` autodiscover provider, we plan to release a `kubernetes` autodiscover one soon. It will give you access to kubernetes metadata. 4. You can add more conditions for them, `templates` setting is a list, so you can put as many as needed ---
**Author:** ![Imran\_Arshad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imran_arshad/32/125900_2.png) [@Imran\_Arshad](https://discuss.elastic.co/u/Imran_Arshad)\ **Post date:** [January 16, 2018, 10:03pm UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/24 "2018-01-16T22:03:10Z")
Hi, I am also trying to do something similar to what Asher is doing. I currently have a prospector that adds Docker metadata to my logs and ships it to Logstash. It looks like this: ``` filebeat: prospectors: - type: log paths: - '/var/lib/docker/containers/*/*.log' json.message_key: log json.keys_under_root: true processors: - add_docker_metadata: ~ ``` What I want to understand is how Autodicover works. Should I replace the prospector with the Autodiscover settings, or does Autodiscover apply to what my prospectors are generating? ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [January 17, 2018, 9:54am UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/25 "2018-01-17T09:54:27Z")
When you use `filebeat.prospectors` you define a static configuration, it will be the same while Filebeat is running. Autodiscover allows you to define conditions and launch different configurations live, based on Autodiscover events from the provider (Docker). If that configuration works for you, you don't need to use autodiscover. If, on the contrary, you want to apply different patterns depending on the container, you may want to define autodiscover rules for that. Best regards ---
**Author:** ![Imran\_Arshad](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/imran_arshad/32/125900_2.png) [@Imran\_Arshad](https://discuss.elastic.co/u/Imran_Arshad)\ **Post date:** [January 17, 2018, 9:19pm UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/26 "2018-01-17T21:19:39Z")
Oh okay. Got it. I will switch to Autodiscover and see how it goes. Thanks Carlos! ---
**Author:** ![Asher\_Shoshan](https://avatars.discourse-cdn.com/v4/letter/a/eada6e/32.png) [@Asher\_Shoshan](https://discuss.elastic.co/u/Asher_Shoshan)\ **Post date:** [January 18, 2018, 8:51am UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/27 "2018-01-18T08:51:37Z")
Hi, I meant, logs lines were transferred to ElasticSearch but not parsed. If I leave only 'access' - then it's ok (but only these lines). In nginx container - they do a symlink from error log to stderr, and from access log to stdout - In terms of Docker, you see only one file ---container-id---.json in var/lib/docker/containers/---container-id-- Asher ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 15, 2018, 8:51am UTC](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759/28 "2018-02-15T08:51:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. [Previous page](https://discuss.elastic.co/t/filebeat-deployment-in-kubernetes-docker/113759.md?page=1)