FIlebeat with services inside of docker compose via autodiscover

Elastic Stack Beats
8

I have reduced the config to this and it works for redis:

docker-compose.yml:

version: "3.3"
  
services:
  redis:
    image: redis:4.0.11-alpine
    healthcheck:
      test: redis-cli ping
      interval: 30s
      timeout: 10s
      retries: 3

  filebeat:
    image: docker.elastic.co/beats/filebeat:6.4.2
    command:
      - "-e"
      - "--strict.perms=false"
    restart: always
    user: root
    volumes:
      - "./filebeat.yml:/usr/share/filebeat/filebeat.yml:ro"
      - "/var/run/docker.sock:/var/run/docker.sock"
      # This is needed for filebeat to load container log path as specified in filebeat.yml
      - "/var/lib/docker/containers/:/var/lib/docker/containers/:ro"
      # This is needed for filebeat to load logs for system and auth modules
      - "/var/log/:/var/log/:ro"

filebeat.yml:

filebeat.autodiscover:
  providers:
    - type: docker
      templates:
        - condition:
            contains:
              docker.container.image: redis
          config:
            - module: redis
              log:
                enabled: true
                input:
                  type: docker
                  containers.ids:
                    - "${data.docker.container.id}"
              slowlog:
                enabled: true
                var.hosts: ["${data.host}:${data.port}"]

output.console:
  pretty: true

Notice that I have added the -e flag to filebeat, this will make it to log to standard output, what is recommended for deployments in docker, and in this case it will also help you to see possible errors.

I have also seen that you have configured elasticsearch:9200 as output, and the elasticsearch service is configured with --transport.host=127.0.0.1. Is it possible that filebeat cannot connect with elasticsearch? You should be able to see it if you add the -e flag mentioned before. If that is the case try to remove this flag, or set it to --transport.host=0.0.0.0.

If this doesn't help and my example doesn't work for you (it should print the events to stdout), check that your docker daemon is configured with the json-file logging driver. You can check it with docker info.