# Filebeat stops collecting docker logs after sometime

**URL:** https://discuss.elastic.co/t/filebeat-stops-collecting-docker-logs-after-sometime/184567
**Category:** Beats
**Tags:** filebeat
**Created:** [June 6, 2019, 11:05am UTC](https://discuss.elastic.co/t/filebeat-stops-collecting-docker-logs-after-sometime/184567 "2019-06-06T11:05:54Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![gauravsachan07](https://avatars.discourse-cdn.com/v4/letter/g/bc8723/32.png) [@gauravsachan07](https://discuss.elastic.co/u/gauravsachan07)
#### Post date: [June 6, 2019, 11:05am UTC](https://discuss.elastic.co/t/filebeat-stops-collecting-docker-logs-after-sometime/184567/1 "2019-06-06T11:05:54Z")

</div>

I have configured filebeat v7.1.0 to autodiscover a particular docker container and send the docker logs to logstash which inturn sends them to elastic search. We are using docker images. When we start the containers it sends the docker logs to logstash properly and same can be seen in logstash docker logs (unfortunately filebeat docker logs is not giving anything) The problem is that sometimes logs stop flowing into logstash and if we re-start filebeat container, it starts pushing the docker container logs again.

Here is the filebeat config yml file entries:  
filebeat.config:  
modules:  
path: ${path.config}/modules.d/\*.yml  
reload.enabled: false

```
filebeat.autodiscover:
  providers:
    - type: docker
      templates:
          - condition:
              contains:
                docker.container.name: container_name
            config:
              - type: docker
                containers.ids:
                  - "${data.docker.container.id}"
                scan_frequency: 10s

processors:
  - add_cloud_metadata: ~

output.logstash:
  hosts: '${LOGSTASH_HOSTS:logstash:5044}'

```

Here is the logstash conf yml file entries

```
input {
  beats {
    port => 5044
  }
}

filter{
        mutate
                {
                        replace => ["message", "%{message}"]
                        gsub => ['message','\r','']
                }
        if [message] =~ "bson-start" {
                grok {
                        match => [ "message",
               "(?<timestamp>%{YEAR}-%{MONTHNUM}-%{MONTHDAY} %{TIME}):%{LOGLEVEL:level} .* bson-start +(?<SisCompleteMessage>.*) bson-end"
             ]
                }
                if [SisCompleteMessage] =~ "instructionId" {
                        json{
                                source => "SisCompleteMessage"
                        }
                }
        }
}

output {

 stdout {
    codec => rubydebug
  }

  elasticsearch {
    hosts => ["elasticsearch:9200"]
  }
}
```

---

<div class="post-metadata">

### Author: ![gauravsachan07](https://avatars.discourse-cdn.com/v4/letter/g/bc8723/32.png) [@gauravsachan07](https://discuss.elastic.co/u/gauravsachan07)
#### Post date: [June 6, 2019, 11:08am UTC](https://discuss.elastic.co/t/filebeat-stops-collecting-docker-logs-after-sometime/184567/2 "2019-06-06T11:08:05Z")

</div>

Here is the docker compose entries for the above containers :

```
version: '3'
services:
  logstash:
    image: logstash:7.1.0
    networks:
      - elknet
    ports:
      - 5044:5044
    volumes:
      - /home/gaursach/elk/logstash_fb_1.conf:/usr/share/logstash/pipeline/logstash.conf
    depends_on:
      - elasticsearch

  filebeat:
    image: docker.elastic.co/beats/filebeat:7.1.0
    user: root
    networks:
      - elknet
    volumes:
      - /home/gaursach/elk/filebeat.docker_1.yml:/usr/share/filebeat/filebeat.yml:ro
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
    command: ["--strict.perms=false"]
    depends_on:
      - logstash

networks:
  elknet:
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 4, 2019, 11:12am UTC](https://discuss.elastic.co/t/filebeat-stops-collecting-docker-logs-after-sometime/184567/3 "2019-07-04T11:12:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.