I have a docker setup with multiple containers running and then setup a filebeat that logs into my elasticsearch. Now I wanted to setup a metricbeat as described here: Set up and run Metricbeat | Metricbeat Reference [7.17] | Elastic
My Elasticseatch is version 7.17. However no matter what I try the integration seems to be faulty. The Dashboards wont work and the Metrics are not mapped or interpreted correctly so that the Metrics Explorer will just show nothing.
This is my metricbeat.yml:
metricbeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
metricbeat.autodiscover:
providers:
- type: docker
hints.enabled: true
metricbeat.modules:
- module: docker
metricsets:
- "container"
- "cpu"
- "diskio"
- "healthcheck"
- "info"
- "image"
- "memory"
- "network"
hosts: [ "unix:///var/run/docker.sock" ]
period: 10s
enabled: true
- module: system
metricsets:
- cpu # CPU usage
- load # CPU load averages
- memory # Memory usage
- network # Network IO
- process # Per process metrics
- process_summary # Process summary
- uptime # System Uptime
- socket_summary # Socket summary
enabled: true
period: 10s
processes: [ '.*' ]
cpu.metrics: [ "percentages","normalized_percentages" ]
core.metrics: [ "percentages" ]
setup.ilm.overwrite: false
setup.ilm.check_exists: false
monitoring.enabled: false
processors:
- add_host_metadata: ~
- add_docker_metadata: ~
- add_fields:
target: ''
fields:
environment: "test-local"
stack: "none"
output.elasticsearch:
hosts: ["https://xxxx.serverurl.com:443/elasticsearch"]
username: metricbeat
password: securepassword
kibana:
host: https://xxxx.serverurl.com:443
username: metricbeat
password: securepassword
And this is my docker compose part:
services:
metricbeat:
image: docker.elastic.co/beats/metricbeat:7.17.13
user: root
volumes:
- ./metricbeat/data:/usr/share/metricbeat/data
- "./metricbeat.yml:/usr/share/metricbeat/metricbeat.yml:ro"
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "/sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro"
- "/proc:/hostfs/proc:ro"
- "/:/hostfs:ro"
The metricbeat logs seem to correctly reach elasticsearch, under the Discover metricbeat index I get tons of entries containing valid information.
However the Dashboard ([Metricbeat Docker] Overview ECS shows a number of running, paused and stopped containers. All the other elements show Error: "[esaggs] > Saved field "container.name" of index pattern "metricbeat-*" is invalid for use with the "Terms" aggregation Please select a new field."
And the Metrics Inventory just has no information what so ever.
When i Go into the Metrics Explorer and select cpu total I get an empty graph.
But I do have values for that.
This is how I started the setup:
docker run -d \
--name=metricbeat \
--user=root \
--volume="./metricbeat.yml:/usr/share/metricbeat/metricbeat.yml:ro" \
--volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
--volume="/sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro" \
--volume="/proc:/hostfs/proc:ro" \
--volume="/:/hostfs:ro" \
docker.elastic.co/beats/metricbeat:7.17.13 metricbeat -e \
-E output.elasticsearch.hosts=["https://xxxx.serverurl.com:443/elasticsearch"]
I also put the false flags from the metricbeat.yml to true for the setup.
This is how I loaded the Dashboards:
docker run docker.elastic.co/beats/metricbeat:7.17.13 setup --dashboards \
-E setup.kibana.host=https://xxxx.serverurl.com:443 \
-E setup.kibana.username=metricbeat \
-E setup.kibana.password=securepasswort
I tryed with the Metricbeat 7.17.12 version, I also went ahead and removed everything related to metricbeat in the elasticsearch I could find and tryed again with the 7.17.13 version. Ran all commands I could find in the Docs but nothing worked. Furthest I got was when the Dashboard showed no errors but countless failed shards.
I am pretty helpless at this point and could really use some help.