# Filebeat not writing docker containers logs into Elasticsearch

**URL:** <https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 24, 2019, 10:32am UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574 "2019-01-24T10:32:22Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![akapit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akapit/32/32490_2.png) [@akapit](https://discuss.elastic.co/u/akapit)\
**Post date:** [January 24, 2019, 10:32am UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/1 "2019-01-24T10:32:23Z")

</div>

Hi,  
I have several docker containers (using docker-compose) and I'm trying to use Filebeat (from docker) to push my containers logs to elasticsearch with no success.

This is my filebeat.yml:

```
filebeat.inputs:
  - type: docker
containers.ids: '*'

output.elasticsearch:
  hosts: ["outflink-elastic:9200"]
  protocol: "http"

```

This is its config block in docker-compose.yml:

```
filebeat:
  image: akapit/outflink_platform:outflink-filebeat
  container_name: outflink-filebeat
  user: root
  volumes:
    - $HOME/filebeat_data:/usr/share/filebeat/data
    - /var/lib/docker/containers:/usr/share/filebeat/dockerlogs:ro
    - /var/run/docker.sock:/var/run/docker.sock
  depends_on:
    - elasticsearch

```

This is my Filebeat Dockerfile:

```
FROM docker.elastic.co/beats/filebeat:6.5.4
COPY filebeat.yml /usr/share/filebeat/filebeat.yml
USER root
RUN chown root:filebeat /usr/share/filebeat/filebeat.yml
RUN mkdir /usr/share/filebeat/dockerlogs

USER filebeat

```

This is the filebeat console output:

```
2019-01-24T10:09:42.348Z	INFO	instance/beat.go:592	Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2019-01-24T10:09:42.354Z	INFO	instance/beat.go:599	Beat UUID: 7e44573c-940f-4ea6-94ab-2673f9962e6c
2019-01-24T10:09:42.354Z	INFO	[seccomp]	seccomp/seccomp.go:116	Syscall filter successfully installed
2019-01-24T10:09:42.354Z	INFO	[beat]	instance/beat.go:825	Beat info	{"system_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "7e44573c-940f-4ea6-94ab-2673f9962e6c"}}}
2019-01-24T10:09:42.356Z	INFO	[beat]	instance/beat.go:834	Build info	{"system_info": {"build": {"commit": "bd8922f1c7e93d12b07e0b3f7d349e17107f7826", "libbeat": "6.5.4", "time": "2018-12-17T20:22:29.000Z", "version": "6.5.4"}}}
2019-01-24T10:09:42.356Z	INFO	[beat]	instance/beat.go:837	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.10.6"}}}
2019-01-24T10:09:42.363Z	INFO	[beat]	instance/beat.go:841	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-01-24T04:34:57Z","containerized":true,"name":"92a6c0af3b08","ip":["127.0.0.1/8","192.168.96.7/20"],"kernel_version":"4.9.125-linuxkit","mac":["02:42:c0:a8:60:07"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":6,"patch":1810,"codename":"Core"},"timezone":"UTC","timezone_offset_sec":0}}}
2019-01-24T10:09:42.364Z	INFO	[beat]	instance/beat.go:870	Process info	{"system_info": {"process": {"capabilities": {"inheritable":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 1, "ppid": 0, "seccomp": {"mode":"filter"}, "start_time": "2019-01-24T10:09:39.920Z"}}}
2019-01-24T10:09:42.365Z	INFO	instance/beat.go:278	Setup Beat: filebeat; Version: 6.5.4
2019-01-24T10:09:42.366Z	INFO	elasticsearch/client.go:163	Elasticsearch url: http://outflink-elastic:9200
2019-01-24T10:09:42.367Z	INFO	[publisher]	pipeline/module.go:110	Beat name: 92a6c0af3b08
2019-01-24T10:09:42.442Z	INFO	instance/beat.go:400	filebeat start running.
2019-01-24T10:09:42.443Z	INFO	[monitoring]	log/log.go:117	Starting metrics logging every 30s
2019-01-24T10:09:42.448Z	INFO	registrar/registrar.go:134	Loading registrar data from /usr/share/filebeat/data/registry
2019-01-24T10:09:42.451Z	INFO	registrar/registrar.go:141	States Loaded from registrar: 36
2019-01-24T10:09:42.452Z	INFO	crawler/crawler.go:72	Loading Inputs: 1
2019-01-24T10:09:42.630Z	INFO	log/input.go:138	Configured paths: [/var/lib/docker/containers/*/*.log]
2019-01-24T10:09:42.630Z	INFO	input/input.go:114	Starting input of type: docker; ID: 8783912846562670036
2019-01-24T10:09:42.630Z	INFO	crawler/crawler.go:106	Loading and starting Inputs completed. Enabled inputs: 1
2019-01-24T10:10:12.486Z	INFO	[monitoring]	log/log.go:144	Non-zero metrics in the last 30s	{"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":260,"time":{"ms":260}},"total":{"ticks":330,"time":{"ms":330},"value":330},"user":{"ticks":70,"time":{"ms":70}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":5},"info":{"ephemeral_id":"cab0928f-44f4-4838-ab5c-d0ab24c8818d","uptime":{"ms":30310}},"memstats":{"gc_next":4194304,"memory_alloc":1821080,"memory_total":4872672,"rss":9252864}},"filebeat":{"events":{"added":56,"done":56},"harvester":{"open_files":0,"running":0}},"libbeat":{"config":{"module":{"running":0}},"output":{"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0,"filtered":56,"total":56}}},"registrar":{"states":{"cleanup":28,"current":8,"update":56},"writes":{"success":56,"total":56}},"system":{"cpu":{"cores":4},"load":{"1":3.25,"15":1.42,"5":1.42,"norm":{"1":0.8125,"15":0.355,"5":0.355}}}}}}

```

As you can see, it doesn't look that there is an error, however nothing gets written in elastic.

I'd appreciate any help,

Thanks!

---

<div class="post-metadata">

**Author:** ![jarpy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jarpy/32/51290_2.png) [@jarpy](https://discuss.elastic.co/u/jarpy)\
**Post date:** [January 24, 2019, 9:53pm UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/2 "2019-01-24T21:53:29Z")

</div>

Hi,

Have you declared a network in `docker-compose.yml`? Could you post the whole file?

Thanks.

---

<div class="post-metadata">

**Author:** ![zaratustra689](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaratustra689/32/43118_2.png) [@zaratustra689](https://discuss.elastic.co/u/zaratustra689)\
**Post date:** [February 6, 2019, 9:53pm UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/3 "2019-02-06T21:53:52Z")

</div>

Hi,

I think you are indenting wrong the yml. It should be:

```auto
filebeat.inputs:
  - type: docker
    containers.ids: '*'
...

```

Regards,

---

<div class="post-metadata">

**Author:** ![akapit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akapit/32/32490_2.png) [@akapit](https://discuss.elastic.co/u/akapit)\
**Post date:** [February 12, 2019, 10:57am UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/4 "2019-02-12T10:57:33Z")

</div>

I've tried that also,... and I don't see any new index.  
Also no errors in filebeat log.

This is my config:

```
filebeat.inputs:

  - type: docker
    containers.ids: '*'

    enabled: true

    json.message_key: log
    json.keys_under_root: true

  filebeat.config.modules:
  # Glob pattern for configuration loading
    path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
    reload.enabled: false

  # Period on which files under path should be checked for changes
    #reload.period: 10s

#==================== Elasticsearch template setting ==========================

setup.template.settings:
  index.number_of_shards: 3
  #index.codec: best_compression
  #_source.enabled: false

#============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["localhost:9200"]

```

This is the log:

```
2019-02-12T10:35:40.643Z	INFO	[seccomp]	seccomp/seccomp.go:116	Syscall filter successfully installed
2019-02-12T10:35:40.643Z	INFO	[beat]	instance/beat.go:936	Beat info	{"system_info": {"beat": {"path": {"config": "/home/outflink/filebeat-6.6.0-linux-x86_64", "data": "/home/outflink/filebeat-6.6.0-linux-x86_64/data", "home": "/home/outflink/filebeat-6.6.0-linux-x86_64", "logs": "/home/outflink/filebeat-6.6.0-linux-x86_64/logs"}, "type": "filebeat", "uuid": "fb60548b-3b65-40c8-8798-0e22acb3fae3"}}}
2019-02-12T10:35:40.643Z	INFO	[beat]	instance/beat.go:945	Build info	{"system_info": {"build": {"commit": "2c385a0764bdc537b6dc078a1d9bf11bb6d7bd95", "libbeat": "6.6.0", "time": "2019-01-24T10:29:55.000Z", "version": "6.6.0"}}}
2019-02-12T10:35:40.643Z	INFO	[beat]	instance/beat.go:948	Go runtime info	{"system_info": {"go": {"os":"linux","arch":"amd64","max_procs":4,"version":"go1.10.8"}}}
2019-02-12T10:35:40.645Z	INFO	[beat]	instance/beat.go:952	Host info	{"system_info": {"host": {"architecture":"x86_64","boot_time":"2019-01-10T10:35:39Z","containerized":false,"name":"localhost","ip":["127.0.0.1/8","139.162.245.198/24","172.17.0.1/16","192.168.144.1/20"],"kernel_version":"4.18.16-x86_64-linode118","mac":["ee:6f:7e:60:16:75","f2:3c:91:5a:0f:e0","02:42:27:0d:79:89","02:42:c4:22:87:f4","6a:59:23:ab:b6:87","82:3e:ea:9a:1b:f2","72:27:43:2b:5d:76","f2:7e:58:bc:68:af","e6:64:b2:99:24:c3","16:0e:49:76:1d:1f","96:c7:89:c6:69:2a","d2:49:67:1b:a0:b2","22:41:d8:dc:f2:f7"],"os":{"family":"debian","platform":"ubuntu","name":"Ubuntu","version":"16.04.5 LTS (Xenial Xerus)","major":16,"minor":4,"patch":5,"codename":"xenial"},"timezone":"UTC","timezone_offset_sec":0,"id":"07c3b878e09b682e46a477335b1ac472"}}}
2019-02-12T10:35:40.645Z	INFO	[beat]	instance/beat.go:981	Process info	{"system_info": {"process": {"capabilities": {"inheritable":null,"permitted":null,"effective":null,"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read"],"ambient":null}, "cwd": "/home/outflink/filebeat-6.6.0-linux-x86_64", "exe": "/home/outflink/filebeat-6.6.0-linux-x86_64/filebeat", "name": "filebeat", "pid": 5984, "ppid": 2545, "seccomp": {"mode":"filter","no_new_privs":true}, "start_time": "2019-02-12T10:35:40.570Z"}}}
2019-02-12T10:35:40.646Z	INFO	instance/beat.go:281	Setup Beat: filebeat; Version: 6.6.0
2019-02-12T10:35:43.656Z	INFO	add_cloud_metadata/add_cloud_metadata.go:319	add_cloud_metadata: hosting provider type not detected.
2019-02-12T10:35:43.656Z	INFO	elasticsearch/client.go:165	Elasticsearch url: http://localhost:9200
2019-02-12T10:35:43.656Z	INFO	[publisher]	pipeline/module.go:110	Beat name: localhost
2019-02-12T10:35:43.657Z	INFO	instance/beat.go:403	filebeat start running.
2019-02-12T10:35:43.657Z	INFO	registrar/registrar.go:134	Loading registrar data from /home/outflink/filebeat-6.6.0-linux-x86_64/data/registry
2019-02-12T10:35:43.657Z	INFO	registrar/registrar.go:141	States Loaded from registrar: 0
2019-02-12T10:35:43.657Z	INFO	crawler/crawler.go:72	Loading Inputs: 1
2019-02-12T10:35:43.657Z	INFO	[monitoring]	log/log.go:117	Starting metrics logging every 30s
2019-02-12T10:35:43.658Z	INFO	log/input.go:138	Configured paths: [/var/lib/docker/containers/*/*.log]
2019-02-12T10:35:43.658Z	INFO	input/input.go:114	Starting input of type: docker; ID: 3192729855305101819
2019-02-12T10:35:43.658Z	INFO	crawler/crawler.go:106	Loading and starting Inputs completed. Enabled inputs: 1
2019-02-12T10:35:43.658Z	INFO	cfgfile/reload.go:150	Config reloader started
2019-02-12T10:35:43.658Z	INFO	cfgfile/reload.go:205	Loading of config files completed.

```

Any ideas?

---

<div class="post-metadata">

**Author:** ![akapit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akapit/32/32490_2.png) [@akapit](https://discuss.elastic.co/u/akapit)\
**Post date:** [February 12, 2019, 10:59am UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/5 "2019-02-12T10:59:56Z")

</div>

No, I have volumes as follows:

```
volumes:
      - $HOME/filebeat_data:/usr/share/filebeat/data
      - /var/lib/docker/containers:/usr/share/filebeat/dockerlogs:ro
      - /var/run/docker.sock:/var/run/docker.sock

```

But anyways I'm currently running filebeat manually from the host machine, to get rid of possible docker configs errors.

---

<div class="post-metadata">

**Author:** ![akapit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akapit/32/32490_2.png) [@akapit](https://discuss.elastic.co/u/akapit)\
**Post date:** [February 12, 2019, 11:01am UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/6 "2019-02-12T11:01:01Z")

</div>

I also tried with the config as exactly described here at the elastic blog:

> **[Enriching logs with Docker metadata using Filebeat](https://www.elastic.co/blog/enrich-docker-logs-with-filebeat)**
>
> Filebeat can now ship Docker logs and enrich them with metadata to make navigating them easier.

With no success..

---

<div class="post-metadata">

**Author:** ![akapit](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/akapit/32/32490_2.png) [@akapit](https://discuss.elastic.co/u/akapit)\
**Post date:** [February 12, 2019, 1:53pm UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/7 "2019-02-12T13:53:54Z")

</div>

I've realised that filebeat wasn't taking the right config file, I'm running it like this:

> sudo ./filebeat -e -c filebeat.yml

and it tries to read the docker logs. Now there is another issue:

`2019-02-12T13:51:10.432Z	ERROR	log/harvester.go:282	Read line error: invalid CRI log format; File: /var/lib/docker/containers/7e8d82f610ae381298e0f5cd8686dbd5d52545cac03f04813d943ae24a1d4773/7e8d82f610ae381298e0f5cd8686dbd5d52545cac03f04813d943ae24a1d4773-json.log`

---

<div class="post-metadata">

**Author:** ![zaratustra689](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zaratustra689/32/43118_2.png) [@zaratustra689](https://discuss.elastic.co/u/zaratustra689)\
**Post date:** [February 12, 2019, 2:13pm UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/8 "2019-02-12T14:13:36Z")

</div>

Hi! So if now `containers.ids: '*'` is correct, I think that these other keys are making filebeat stop:

```auto
    json.message_key: log
    json.keys_under_root: true

```

They need to be indented at the same level as `ids`. Try the following:

```auto
filebeat.inputs:
  - type: docker
    containers:
        ids:
             - '*'
        json.message_key: log
        json.keys_under_root: true

```

It happend to me some days ago and that solved my problem. (Pay attention to the indentation)

Cheers!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 12, 2019, 2:13pm UTC](https://discuss.elastic.co/t/filebeat-not-writing-docker-containers-logs-into-elasticsearch/165574/9 "2019-03-12T14:13:43Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.