# Filebeat on "Docker for AWS" fails to start **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 5, 2017, 12:31pm UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345 "2017-12-05T12:31:20Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![shazChaudhry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shazchaudhry/32/25027_2.png) [@shazChaudhry](https://discuss.elastic.co/u/shazChaudhry)\ **Post date:** [December 5, 2017, 12:31pm UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345/1 "2017-12-05T12:31:20Z")
My Elastic stack version is at 6.0.0; ELK + Filebeat + Metricbeat running in docker swarm mode; Docker host machines are all ubuntu/xenial64. I have configured filebeat as described in this blog by Carlos Pérez-Aradros: [https://www.elastic.co/blog/enrich-docker-logs-with-filebeat](https://www.elastic.co/blog/enrich-docker-logs-with-filebeat) which works fine and I can see data in Kibana. ```auto filebeat.prospectors: - type: log paths: - '/var/lib/docker/containers/*/*.log' json.message_key: log json.keys_under_root: true processors: - add_docker_metadata: ~ output.elasticsearch: hosts: ["elasticsearch:9200"] ``` Now, has anyone tried getting filebeat working on "docker for AWS (CE)"? '/var/lib/docker/containers' can not be mounted and so filebeat fails to start. On any node in "Docker for AWS", I get this error: - ls: cannot access /var/lib/docker/containers: No such file or directory for anyone who would like to try, here are my docker compose and filebeat.yml files - [https://github.com/shazChaudhry/docker-elastic/blob/master/filebeat-docker-compose.yml](https://github.com/shazChaudhry/docker-elastic/blob/master/filebeat-docker-compose.yml) - [https://github.com/shazChaudhry/docker-elastic/blob/master/elk/beats/filebeat/config/filebeat.yml](https://github.com/shazChaudhry/docker-elastic/blob/master/elk/beats/filebeat/config/filebeat.yml) I will be grateful if anyone give us a shout if they have got filebeat working on "Docker for AWS" ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [December 7, 2017, 4:40am UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345/2 "2017-12-07T04:40:20Z")
Never tested on AWS but I wonder where AWS CE stores the log files of the containers? Or do the directories exist and it's just an permission problem? ---
**Author:** ![exekias](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/exekias/32/28718_2.png) [@exekias](https://discuss.elastic.co/u/exekias)\ **Post date:** [December 7, 2017, 12:23pm UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345/3 "2017-12-07T12:23:09Z")
It seems Docker for AWS uses it's own logging driver, probably you think you can switch that to use `json-file`? [https://docs.docker.com/engine/admin/logging/json-file/](https://docs.docker.com/engine/admin/logging/json-file/) ---
**Author:** ![shazChaudhry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shazchaudhry/32/25027_2.png) [@shazChaudhry](https://discuss.elastic.co/u/shazChaudhry)\ **Post date:** [December 7, 2017, 11:20pm UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345/4 "2017-12-07T23:20:18Z")
According to this issue on [Docker4AWS Github](https://github.com/docker/for-aws/issues/123): > when you ssh into a Docker4AWS instance, you are inside a container called shell hosting the SSH server. The shell container does not have the necessary privileges required for mounting a device. Well, I will need to try [--pid](https://docs.docker.com/engine/reference/run/#pid-settings-pid) option to see if directories from the host can be picked up ---
**Author:** ![shazChaudhry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shazchaudhry/32/25027_2.png) [@shazChaudhry](https://discuss.elastic.co/u/shazChaudhry)\ **Post date:** [December 7, 2017, 11:50pm UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345/5 "2017-12-07T23:50:27Z")
I have tried [--pid](https://docs.docker.com/engine/reference/run/#pid-settings-pid) option but I still get this error: > "invalid mount config for type "bind": bind source path does not exist" I will be grateful if anyone can suggest how Filebeat 6.0.0 can be started in swarm mode on Docker for AWS. ---
**Author:** ![shazChaudhry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shazchaudhry/32/25027_2.png) [@shazChaudhry](https://discuss.elastic.co/u/shazChaudhry)\ **Post date:** [December 17, 2017, 9:25pm UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345/6 "2017-12-17T21:25:16Z")
@exekias @ruflin - This particular issue is resolved and I am pleased to report that Filebeat does work as a global service in docker swarm mode on "_Docker for AWS_" There were actually two problems with my configuration that prevented Filebeat from starting: 1. I had to turn off Auditd Filebeat module in the config file as it turned out that Alpine Linux does not have Auditd package. So, in short, it is not possible to start Auditd module of Filebeat on "_Docker for AWS_". 2. To get Filebeat working, I had to set "_host_" to add\_docker\_metadata processor ```auto filebeat.prospectors: - type: log enabled: true paths: - '/var/lib/docker/containers/*/*.log' json.message_key: log json.add_error_key: true json.keys_under_root: true processors: - add_docker_metadata: host: "unix:///var/run/docker.sock" ``` For anyone wishing to take a look at my docker compose and Filebeat config files, here they are on GitHub: - [https://github.com/shazChaudhry/docker-elastic/blob/248d776e7f6dfd10d9c6c703bcd846a825028e7c/elk/beats/filebeat/config/filebeat.yml](https://github.com/shazChaudhry/docker-elastic/blob/248d776e7f6dfd10d9c6c703bcd846a825028e7c/elk/beats/filebeat/config/filebeat.yml) - [https://github.com/shazChaudhry/docker-elastic/blob/248d776e7f6dfd10d9c6c703bcd846a825028e7c/filebeat-docker-compose.yml](https://github.com/shazChaudhry/docker-elastic/blob/248d776e7f6dfd10d9c6c703bcd846a825028e7c/filebeat-docker-compose.yml) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 14, 2018, 9:36pm UTC](https://discuss.elastic.co/t/filebeat-on-docker-for-aws-fails-to-start/110345/7 "2018-01-14T21:36:48Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.