# Filebeat autodiscovery for docker seems to miss collecting logs of crashed containers

**URL:** <https://discuss.elastic.co/t/filebeat-autodiscovery-for-docker-seems-to-miss-collecting-logs-of-crashed-containers/159324>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [December 4, 2018, 9:36am UTC](https://discuss.elastic.co/t/filebeat-autodiscovery-for-docker-seems-to-miss-collecting-logs-of-crashed-containers/159324 "2018-12-04T09:36:27Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![farodin91](https://avatars.discourse-cdn.com/v4/letter/f/977dab/32.png) [@farodin91](https://discuss.elastic.co/u/farodin91)\
**Post date:** [December 4, 2018, 9:36am UTC](https://discuss.elastic.co/t/filebeat-autodiscovery-for-docker-seems-to-miss-collecting-logs-of-crashed-containers/159324/1 "2018-12-04T09:36:27Z")

</div>

Hi

We are running a multi-node swarm. If services crashes and produces a log entry with the crash exception, these logs are not forward to our Logstash. Besides, we are able to see these logs with docker log.

Version:  
Filebeat: [docker.elastic.co/beats/filebeat:6.5.1](http://docker.elastic.co/beats/filebeat:6.5.1)  
Docker: 18.09

Config:

filebeat.yml

```
logging.metrics.enabled: false

filebeat.registry_file: ${path.data}/registry

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml

filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

fields:
  env: ${swarm.environment}

output.logstash:
  hosts: ["${logstash.url}:${logstash.port}"]
  slow_start: true

```

docker-compose.yml

```
version: '3.2'

services:
  logstash:
    image: logstash_image
    volumes:
      - /usr/share/logstash/queue/:/usr/share/logstash/queue/
    deploy:
      mode: replicated
      replicas: 1

  filebeat:
    image: logstash_image
    volumes:
     - /var/lib/docker/containers/:/var/lib/docker/containers/:ro
     - /var/run/docker.sock:/var/run/docker.sock:ro
     - /usr/share/filebeat/data/:/usr/share/filebeat/data/
     - /etc/hostname:/etc/hostname:ro
     - /var/log/:/var/log/:ro
    environment:
      swarm.environment: develop
    deploy:
      mode: global

networks:
  default:

```

Why we are not seeing these logs?

Dear Jan

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [December 5, 2018, 1:52pm UTC](https://discuss.elastic.co/t/filebeat-autodiscovery-for-docker-seems-to-miss-collecting-logs-of-crashed-containers/159324/2 "2018-12-05T13:52:50Z")

</div>

Which modules are you running?

Have you checked filebeat logs for errors?

Have you checked if filebeat is reading the log file (registry file contains offset, log includes info message on Start/Stop of a harvester)?

---

<div class="post-metadata">

**Author:** ![farodin91](https://avatars.discourse-cdn.com/v4/letter/f/977dab/32.png) [@farodin91](https://discuss.elastic.co/u/farodin91)\
**Post date:** [December 5, 2018, 3:02pm UTC](https://discuss.elastic.co/t/filebeat-autodiscovery-for-docker-seems-to-miss-collecting-logs-of-crashed-containers/159324/3 "2018-12-05T15:02:36Z")

</div>

Which modules are you running?

Only system and docker autodiscover

Have you checked filebeat logs for errors?

There is one error which is already report and fixed in the master [https://github.com/elastic/beats/pull/9305](https://github.com/elastic/beats/pull/9305)

Have you checked if filebeat is reading the log file (registry file contains offset, log includes info message on Start/Stop of a harvester)?

I see only logs up to the registry position.

| date | level | path | messsage |
| --- | --- | --- | --- |
| 2018-12-05T14:41:57.938Z | INFO | log/input.go:138 | Configured paths: [/var/lib/docker/containers/02da23a669acb638c061b582999f0a9262e01fce1d2e2624ab745f22c2902b48/\*.log] |
| 2018-12-05T14:41:57.938Z | INFO | input/input.go:114 | Starting input of type: docker; ID: 11189854344855006298 |
| 2018-12-05T14:41:57.938Z | INFO | log/harvester.go:254 | Harvester started for file: /var/lib/docker/containers/02da23a669acb638c061b582999f0a9262e01fce1d2e2624ab745f22c2902b48/02da23a669acb638c061b582999f0a9262e01fce1d2e2624ab745f22c2902b48-json.log |
| 2018-12-05T14:43:13.375Z | INFO | input/input.go:149 | input ticker stopped |
| 2018-12-05T14:43:13.375Z | INFO | input/input.go:167 | Stopping Input: 11189854344855006298 |
| 2018-12-05T14:43:13.375Z | INFO | log/harvester.go:275 | Reader was closed: /var/lib/docker/containers/02da23a669acb638c061b582999f0a9262e01fce1d2e2624ab745f22c2902b48/02da23a669acb638c061b582999f0a9262e01fce1d2e2624ab745f22c2902b48-json.log. Closing. |

---

<div class="post-metadata">

**Author:** ![farodin91](https://avatars.discourse-cdn.com/v4/letter/f/977dab/32.png) [@farodin91](https://discuss.elastic.co/u/farodin91)\
**Post date:** [December 27, 2018, 2:48pm UTC](https://discuss.elastic.co/t/filebeat-autodiscovery-for-docker-seems-to-miss-collecting-logs-of-crashed-containers/159324/4 "2018-12-27T14:48:07Z")

</div>

Any progress?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 24, 2019, 2:48pm UTC](https://discuss.elastic.co/t/filebeat-autodiscovery-for-docker-seems-to-miss-collecting-logs-of-crashed-containers/159324/5 "2019-01-24T14:48:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.