# Filtering setup for docker containers not working

**URL:** https://discuss.elastic.co/t/filtering-setup-for-docker-containers-not-working/295573
**Category:** Beats
**Tags:** docker, filebeat
**Created:** [January 27, 2022, 11:27am UTC](https://discuss.elastic.co/t/filtering-setup-for-docker-containers-not-working/295573 "2022-01-27T11:27:52Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![mitlonik](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mitlonik](https://discuss.elastic.co/u/mitlonik)
#### Post date: [January 27, 2022, 11:27am UTC](https://discuss.elastic.co/t/filtering-setup-for-docker-containers-not-working/295573/1 "2022-01-27T11:27:52Z")

</div>

filtering setup for docker containers not working.

I started Elastic Stack without anybody additional settings. And here are my filebeat settings:  
filebeat.yml

```auto
filebeat.autodiscover:
  providers:
    - type: docker
      labels.dedot: true
      hints.enabled: true
      templates:
        - condition:
            contains:
              container.labels.collect_logs_with_filebeat: "true" # label name in service docker-compose file
              docker.container.name: "test_golang_app_2"
          config:
            - type: container
              format: docker # auto, docker, cli
              # stream: stdout # all, stdout, stderr
              #containers.ids:
              # - "${data.docker.container.id}"
              paths:
                - "/var/lib/docker/containers/${data.docker.containers.id}/*.log"

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1

setup.kibana:
  host: "localhost:5601"

output.elasticsearch:
  enabled: true
  hosts: ["localhost:9200"]

output.logstash:
  enabled: false
  hosts: ["localhost:5044"]

processors:
  - drop_fields:
      fields: ["agent.ephemeral_id", "agent.hostname", "agent.id", "agent.name", "agent.version", "docker.container.labels.com_docker_compose_config-hash", "docker.container.labels.com_docker_compose_container-number", "docker.contain>
      ignore_missing: false

monitoring.enabled: false
logging.metrics.enabled: false
logging.level: debug
logging.selectors: ["*"]
logging.to_files: true

```

and to test, I wrote a simple application in docker container that constantly sends data to the stdout stream

docker-compose.yml

```auto
version: '3.7'

services:
  simple_golang_app:
    image: simple_golang_app
    container_name: simple_golang_app
    build:
      context: app/golang/
      dockerfile: Dockerfile
      args:
        TEST_ENV: $TEST_ENV
    networks:
      - net

  test_golang_app_2:
    image: test_golang_app_2
    container_name: test_golang_app_2
    build:
      context: app/golang/
      dockerfile: Dockerfile
      args:
        TEST_ENV: $TEST_ENV
    networks:
      - net
    deploy:
      labels:
        docker.container.labels.description: "collect_logs_with_filebeat"
        co.elastic.logs/enabled: "true" # for Filebeat
        collect_logs_with_filebeat: "true"
    labels:
        docker.container.labels.description: "collect_logs_with_filebeat"
        co.elastic.logs/enabled: "true" # for Filebeat
        collect_logs_with_filebeat: "true"

networks:
  net:
    driver: overlay

```

main.go

```auto
package main

import (
        "fmt"
		"io"
        "os"
		"time" // https://pkg.go.dev/time
)

func main() {
        fmt.Println("Print from the Go program")
        fmt.Println(os.Getenv("TEST_ENV"))

		io.WriteString(os.Stdout,"This is the line to standard output.\n")
		io.WriteString(os.Stderr,"This is the line for standard error output.\n")

		// print every 5 seconds how long the program is running
		for range time.Tick(time.Second * 30) {
			go func() {
				fmt.Println(os.Stdout, time.Now())
			}()
		}
}

```

Dockerfile

```auto
FROM golang:1.17-alpine

ADD main.go /home

WORKDIR /home

RUN \
    apk add --no-cache bash git openssh && \
    go get -u github.com/minio/minio-go

CMD ["go","run","main.go"]

```

The problem is that the templates filter doesn't work. I have tried various methods: by container name, by adding labels, but I still see the logs of all running containers on my host, and not a specific one.  
How can I still configure the sending of logs for certain containers, and not all those running on the host?

---

<div class="post-metadata">

### Author: ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)
#### Post date: [January 28, 2022, 10:53am UTC](https://discuss.elastic.co/t/filtering-setup-for-docker-containers-not-working/295573/2 "2022-01-28T10:53:07Z")

</div>

Hi,

what do you mean by "templates filter doesn't work"? Did you see any error or just filtering is not applied? I'm wondering if debug mode can reveal more details.

BTW which version of Elastic stack and filebeat are you using?

---

<div class="post-metadata">

### Author: ![mitlonik](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mitlonik](https://discuss.elastic.co/u/mitlonik)
#### Post date: [January 28, 2022, 12:38pm UTC](https://discuss.elastic.co/t/filtering-setup-for-docker-containers-not-working/295573/3 "2022-01-28T12:38:16Z")

</div>

Thanks for the answer!

Yes, fitering is not applied.  
I'm use 7.16.3.

I have the debug option set, but there is nothing in the filebeat logs about processing templates in them.

filebeat.yml

```auto
......
logging.level: debug
......

```

maybe I completely misunderstand how filebeat templates work?

the full code of my Elastic Stack is here - [GitHub - mitlonik/elastic\_stack: everything is working. Filebeat finds all running docker containers on the host and sends the stdout stream from the containers to logstash or elasticsearch (depending on needs)](https://github.com/mitlonik/elastic_stack)  
a simple application that will help test this problem is here - [GitHub - mitlonik/testing\_applications: Applications in different programming languages for testing various tasks](https://github.com/mitlonik/testing_applications)

---

<div class="post-metadata">

### Author: ![mitlonik](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mitlonik](https://discuss.elastic.co/u/mitlonik)
#### Post date: [January 31, 2022, 6:27am UTC](https://discuss.elastic.co/t/filtering-setup-for-docker-containers-not-working/295573/4 "2022-01-31T06:27:14Z")

</div>

Please tell me, do you have any other ideas about this?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 28, 2022, 8:27am UTC](https://discuss.elastic.co/t/filtering-setup-for-docker-containers-not-working/295573/5 "2022-02-28T08:27:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
