# Filebeat 6.6.2 and autodiscover docker log harversting

**URL:** <https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 3, 2019, 7:09pm UTC](https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267 "2019-04-03T19:09:27Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![bmmpt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bmmpt/32/43439_2.png) [@bmmpt](https://discuss.elastic.co/u/bmmpt)\
**Post date:** [April 3, 2019, 7:09pm UTC](https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267/1 "2019-04-03T19:09:27Z")

</div>

On a Windows 10 OS, I have a series of docker containers running java applications. Each container internally logs to:

/usr/app/logs/some\_log\_file\_name.log

A sample log entry would look like:

`{"timestamp":"2019-04-02T21:28:16.786Z","level":"INFO","machineName":"api-query","appVersion":"${env:LOGGER_APP_VERSION}","appName":"${env:LOGGER_APP_NAME}","logger":"com.lw.spring.api.common.services.CacheService","message":"Region added: permission-cache","thread":"main"}`

I wish to be able to visualize those logs in kibana, but haven't been able to get the right configuration going. I'm using filebeat 6.6.2.

Here's what I have now in my filebeat.yml file:

```
filebeat.autodiscover:
  providers:
    - type: docker
      host: "tcp://127.0.0.1:2375"
      templates:
        - condition:
            contains:
              docker.container.image: docker-repo
          config:
            - type: docker
              containers:
                ids:
                    - "${data.docker.container.id}"
               
filebeat.inputs:
- type: docker
  enabled: true
  containers:
    ids:
      - "*"
    processors:
      - add_docker_metadata:

```

When I start the service, there are no ERROR logs on the filebeat log file. Aside from not getting any logs in Kibana I also notice in the logs:

1. Log entry:  
2019-04-03T15:00:05.486-0400 DEBUG [bus] bus/bus.go:72 filebeat: map[start:true host:172.17.0.2 port:6379 docker:{"container":{"id":"d74054963f2e757e66285725e4e5a8e9ebe65f40a32d72dbd0c0b2e25922f593","image":"redis"

Why would it be picking up the "redis" image when I specified the image should contain "docker-repo"?

1. Log entry:  
2019-04-03T15:00:05.487-0400 INFO log/input.go:138 Configured paths: [C:\var\lib\docker\containers\d1837b011439a0463b5727ea71317a4e2e674c950321f30acb5511cd305a075c\*.log]

What is the meaning of these kind of paths? Are these supposed to be paths on the windows host or inside the docker containers?

Appreciate any help I can get,  
Bruno

---

<div class="post-metadata">

**Author:** ![bmmpt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bmmpt/32/43439_2.png) [@bmmpt](https://discuss.elastic.co/u/bmmpt)\
**Post date:** [April 29, 2019, 6:26pm UTC](https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267/2 "2019-04-29T18:26:57Z")

</div>

No replies? No direction? Am I way off here? 😃

---

<div class="post-metadata">

**Author:** ![Debashis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/debashis/32/45290_2.png) [@Debashis](https://discuss.elastic.co/u/Debashis)\
**Post date:** [April 30, 2019, 4:32am UTC](https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267/3 "2019-04-30T04:32:08Z")

</div>

@bmmpt, As you have configured both autodiscover provider and docker type input so you are getting each and every container with all images rather than only "docker-repo"

You have configured "docker-repo" under autodiscover provider but in "filebeat.inputs" type "docker" you have mentioned the containers.ids as "\*", so "redis" image gets picked.

There is no such importance to use docker autodiscover provider with filebeat.inputs of type docker as you can mention it under "config" parameter of autdiscover provider.

The indentation of "processor" in your config file is not correct as well.

Filebeat docker type input and autodiscover provider consider docker's log under "/var/lib/docker/contianers/" by default. So as you have not mentioned any container's log path so it is taking default path, for this reason your logs are not getting harvested. So you have to configure "path" / "paths" as below

```
filebeat.autodiscover:
  providers:
    - type: docker
      host: "tcp://127.0.0.1:2375"
      templates:
        - condition:
            contains:
              docker.container.image: docker-repo
          config:
            - type: docker
              containers:
                ids:
                    - "${data.docker.container.id}"
                paths:
                    - /usr/app/logs/*
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 28, 2019, 4:32am UTC](https://discuss.elastic.co/t/filebeat-6-6-2-and-autodiscover-docker-log-harversting/175267/4 "2019-05-28T04:32:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.