Filebeat 7.3.0 Autodiscover configuration introduction yields "Cannot connect to the Docker daemon" error

Elastic Stack Beats
,
1

I would like to roll out an ELK stack using filebeat to transport logs from custom docker containers. The OS is Ubuntu 18.04.

I am trying to do this by using the filebeat autodiscover feature by following the documentation:

https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover.html

In the documentation is states that you should just add a filebeat.autodiscover field to the yaml file.

Here is what I have done:

  1. Run the docker container and copy the default config
  2. Add in the filebeat.autodiscover in the documentation

If I run with the filebeat.yaml file containing the filebeat.autodiscover field, it does not run and gives the following error:

2019-08-06T09:46:40.913Z	ERROR	instance/beat.go:877	Exiting: error in autodiscover provider settings: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Exiting: error in autodiscover provider settings: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Here is my filebeat.yaml config file:

filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

filebeat.autodiscover:
  providers:
    - type: docker
      labels.dedot: true
      templates:
        - condition:
            contains:
              docker.container.image: redis
          config:
            - type: docker
              containers.ids:
                - "${data.docker.container.id}"
              exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines

processors:
- add_cloud_metadata: ~

output.elasticsearch:
  hosts: '${ELASTICSEARCH_HOSTS:elasticsearch:9200}'
  username: '${ELASTICSEARCH_USERNAME:}'
  password: '${ELASTICSEARCH_PASSWORD:}'

And i am running the command:

$ docker run -v ~/elastic/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml docker.elastic.co/beats/filebeat:7.3.0

where ~/elastic/filebeat/filebeat.yml contains the config

When I remove the filebeat.autodiscover section, the config runs without the error.

Thank you for any suggestions

2

is user used to run filebeat part of your docker group?

3

I dont specify a --user argument to docker, just the command posted above.
Is this necessary, and if so why?

4

my thinking was that filebeat is not able to connect to docker daemon to subscribe for events.
if this is the case, please check if

  • docker is running
  • user which is used for running filebeat has access rights for /var/run/docker.sock

usually /var/run/docker.sock is accessible for owner and a group where owner is a root and group set to docker. check whether user used to run filebeat is member of a docker group

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.