# Add\_docker\_metadata failing to get container id if logs are in different path

**URL:** https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452
**Category:** Beats
**Tags:** filebeat
**Created:** [August 8, 2018, 8:22am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452 "2018-08-08T08:22:42Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![somelifeDev](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@somelifeDev](https://discuss.elastic.co/u/somelifeDev)
#### Post date: [August 8, 2018, 8:22am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452/1 "2018-08-08T08:22:42Z")

</div>

Setup:

- docker logs are not in the default folder "/var/lib/docker/containers/_/_-json.log" but in "/home/var/docker/data/containers/_/_-json.log"
- Filebeat works just fine and pushes docker logs into elasticsearch

Problem:

- "docker.container.id" always says "containers" instead of the actual ids
- This leads to "add\_docker\_metadata:" errors, since the container "containers" doesn't exist

It's as if, when trying to get the id of the containers, it's trying to get the 5th item in the path, but since in my case it's the 6th, it just gets "containers" (the actual 5th item in the path).

Is this hardcoded somewhere or something? Any idea how to solve this?

---

<div class="post-metadata">

### Author: ![somelifeDev](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@somelifeDev](https://discuss.elastic.co/u/somelifeDev)
#### Post date: [August 8, 2018, 8:30am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452/2 "2018-08-08T08:30:03Z")

</div>

As a note, I added the path in the containers struct definition, which gives me this as input for the filebeat.yml

```
- type: docker
  enabled: true
  containers:
    ids:
      - "*"
    path: "/home/var/docker/data/containers"
  scan_frequency: 10s
  processors:
  - add_docker_metadata:
      host: "unix:///var/run/docker.sock"
```

---

<div class="post-metadata">

### Author: ![somelifeDev](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@somelifeDev](https://discuss.elastic.co/u/somelifeDev)
#### Post date: [August 8, 2018, 8:40am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452/3 "2018-08-08T08:40:50Z")

</div>

And issue is the same with input:log

```
- type: log
  enabled: true
  paths:
    - /home/var/docker/data/containers/*/*-json.log
  scan_frequency: 10s
  processors:
  - add_docker_metadata:
      host: "unix:///var/run/docker.sock"

```

"docker.container.id" is always "containers"  
and in the logs for filebeat, there is this all the time

> 2018-08-08T10:38:29.730+0200 DEBUG [add\_docker\_metadata] add\_docker\_metadata/add\_docker\_metadata.go:169 Container not found: cid=containers

---

<div class="post-metadata">

### Author: ![somelifeDev](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@somelifeDev](https://discuss.elastic.co/u/somelifeDev)
#### Post date: [August 8, 2018, 9:10am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452/4 "2018-08-08T09:10:38Z")

</div>

Ok, I found the problem

It's HARDCODED that ids should be in the 5th position of the path..

see [github](https://github.com/elastic/beats/blob/35a7882f69d6daf9ab72f36c6509a00aaa2904d0/libbeat/processors/add_docker_metadata/config.go)

---

<div class="post-metadata">

### Author: ![somelifeDev](https://avatars.discourse-cdn.com/v4/letter/s/5fc32e/32.png) [@somelifeDev](https://discuss.elastic.co/u/somelifeDev)
#### Post date: [August 8, 2018, 9:27am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452/5 "2018-08-08T09:27:13Z")

</div>

alright, my bad. Checking the source more closely, you can actually inject parameters for this using not sourceindex but match\_source\_index, like you can change the host.

```
 type: log
  enabled: true
  paths:
    - /home/var/docker/data/containers/*/*-json.log
  scan_frequency: 10s
  processors:
  - add_docker_metadata:
      match_source_index: 5

```

close this thread whenever

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [August 10, 2018, 6:30am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452/6 "2018-08-10T06:30:58Z")

</div>

Glad you found a solution and thanks for sharing it here.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 7, 2018, 6:30am UTC](https://discuss.elastic.co/t/add-docker-metadata-failing-to-get-container-id-if-logs-are-in-different-path/143452/7 "2018-09-07T06:30:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
