# Running Metricbeat in Docker on ECS, want to monitor the other Docker images also running on the same cluster

**URL:** <https://discuss.elastic.co/t/running-metricbeat-in-docker-on-ecs-want-to-monitor-the-other-docker-images-also-running-on-the-same-cluster/167369>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [February 6, 2019, 11:18pm UTC](https://discuss.elastic.co/t/running-metricbeat-in-docker-on-ecs-want-to-monitor-the-other-docker-images-also-running-on-the-same-cluster/167369 "2019-02-06T23:18:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ppafford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppafford/32/36533_2.png) [@ppafford](https://discuss.elastic.co/u/ppafford)\
**Post date:** [February 6, 2019, 11:18pm UTC](https://discuss.elastic.co/t/running-metricbeat-in-docker-on-ecs-want-to-monitor-the-other-docker-images-also-running-on-the-same-cluster/167369/1 "2019-02-06T23:18:18Z")

</div>

the goal: Try to monitor other docker containers on ECS cluster

info: metricbeats is also running in docker on the same ECS cluster

```
metricbeat.config.modules:
  path: ${path.config}/conf.d/*.yml
  reload.period: 10s
  reload.enabled: true

metricbeat.max_start_delay: 10s
  
metricbeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
    - type: docker
      templates:
        - condition:
            contains:
              docker.container.image: "docker-appname-1"
          config:
            - module: apache
              metricsets: ["status"]
              period: 10s
              # hosts: ["${data.host:localhost}"]
              hosts: ["${data.host}"] ## no hostname being passed
              server_status_path: "version"
    - type: docker
      templates:
        - condition:
            contains:
              docker.container.image: "docker-appname-2"
          config:
            - module: apache
              metricsets: ["status"]
              period: 10s
              # hosts: ["${data.host:localhost}"]
              hosts: ["${data.host}"] ## no hostname being passed
              server_status_path: "version"
    - type: docker
      templates:
        - condition:
            contains:
              docker.container.image: "docker-appname-3"
          config:
            - module: system
              metricsets: ["cpu", "memory"]
              period: 10s
              # hosts: ["${data.host:localhost}"]
              hosts: ["${data.host}"] ## no hostname being passed

output.elasticsearch:
  hosts: ["remote.es.instance.url:9200"]

## NOTE: if this is enabled the container does not start on ECS
# output.console:
# codec.json:
# pretty: true
# escape_html: false

path.data: /var/log/metricbeat/data
path.logs: /var/log/metricbeat/logs

setup.dashboards:
  enabled: true

setup.template:
  overwrite: true
  enabled: true

setup.kibana:
  host: "remote.kibana.instance.ur:443"
  protocol: "https"
  username: "user"
  password: "pass"
  ssl.enabled: true
  ssl.verification_mode: "none"

logging.level: debug
logging.selectors: ["*"]
logging.to_syslog: false
logging.to_eventlog: false
logging.metrics.enabled: true
logging.metrics.period: 30s
logging.to_files: true
logging.files:
  path: /var/log/metricbeat
  name: metricbeat.log
  keepfiles: 7
  rotateeverybytes: 10485760 # 10 MB
  permissions: 0664
logging.json: false

```

I have this in my docker.yml module

```
- module: docker
  metricsets:
    - container
    - cpu
    - diskio
  # - healthcheck
    - info
    - memory
    - network
  period: 10s
  hosts: ["unix:///var/run/docker.sock"]

```

I see the dashboards created in kibana, I see the metricbeat docker container in the infrastructure tab, I see data coming from metricbeat

example data in kibana, changed some values but structure is same

```
{
  "_index": "metricbeat-6.5.4-2019.02.06",
  "_type": "doc",
  "_id": “12121kj1b3kj12b3khbk2”,
  "_version": 1,
  "_score": null,
  "_source": {
    "@timestamp": "2019-02-06T23:11:47.203Z",
    "metricset": {
      "rtt": 1292476,
      "name": "status",
      "module": "apache",
      "host": “hostname” ## I’ve tried to manually set this as well
    },
    "apache": {
      "status": {
        "cpu": {},
        "connections": {
          "async": {}
        },
        "hostname": "hostname",
        "scoreboard": {
          "logging": 0,
          "total": 0,
          "closing_connection": 0,
          "gracefully_finishing": 0,
          "idle_cleanup": 0,
          "waiting_for_connection": 0,
          "starting_up": 0,
          "reading_request": 0,
          "sending_reply": 0,
          "open_slot": 0,
          "keepalive": 0,
          "dns_lookup": 0
        },
        "workers": {},
        "uptime": {}
      }
    },
    "docker": {
      "container": {
        "id": “askjd9823hdnnajnsbxasnxo9jo2nij3ndjknxnasx”, ## this is the container id for metricbeat and not the host I’m trying to monitor
        "name": "ecs-appname-5-env-fjaksdnbcoun901",
        "image": “ajsndoijned.amazonaws.com/appname:latest",
        "labels": {
          "com": {
            "amazonaws": {
              "ecs": {
                "task-definition-family": “app name”,
                "task-definition-version": "5",
                "cluster": “cluster name”,
                "container-name": “container name of app”,
                "task-arn": "arn:…:task/kajsndakjnsd-asdasdnkajnx-asdasda”
              }
            }
          },
          "version": "1.0.0",
          "environment": "dev",
          "applicationContact": “app developer”,
          "applicationName": “app name”,
          "awsAdminContact": “ops”
        }
      }
    },
    "beat": {
      "name": "Metricbeat.local",
      "hostname": “host-Metricbeat.local",
      "version": "6.5.4"
    },
    "host": {
      "name": "host-Metricbeat.local"
    }
  },
  "fields": {
    "@timestamp": [
      "2019-02-06T23:11:47.203Z"
    ]
  },
  "highlight": {
    "docker.container.id": [
      "@kibana-highlighted-field@f116kjasbdkaljsdnajsdkja06f1edeee23b0939d3@/kibana-highlighted-field@"
    ]
  },
  "sort": [
    1549494707203
  ]
}
```

---

<div class="post-metadata">

**Author:** ![ppafford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppafford/32/36533_2.png) [@ppafford](https://discuss.elastic.co/u/ppafford)\
**Post date:** [February 7, 2019, 4:58pm UTC](https://discuss.elastic.co/t/running-metricbeat-in-docker-on-ecs-want-to-monitor-the-other-docker-images-also-running-on-the-same-cluster/167369/2 "2019-02-07T16:58:32Z")

</div>

> [@ppafford](#):
>
> metricbeat.config.modules: path: ${path.config}/conf.d/\*.yml reload.period: 10s reload.enabled: true

this was the issue, needed to be `path: ${path.config}/modules.d/*.yml`

---

<div class="post-metadata">

**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\
**Post date:** [February 8, 2019, 11:08am UTC](https://discuss.elastic.co/t/running-metricbeat-in-docker-on-ecs-want-to-monitor-the-other-docker-images-also-running-on-the-same-cluster/167369/3 "2019-02-08T11:08:54Z")

</div>

Oh! thank you for posting! I'm glad that you fixed it 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 8, 2019, 11:08am UTC](https://discuss.elastic.co/t/running-metricbeat-in-docker-on-ecs-want-to-monitor-the-other-docker-images-also-running-on-the-same-cluster/167369/4 "2019-03-08T11:08:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.