Metricbeat 7.13 not working

I have a docker setup with multiple containers running and then setup a filebeat that logs into my elasticsearch. Now I wanted to setup a metricbeat as described here: Set up and run Metricbeat | Metricbeat Reference [7.17] | Elastic
My Elasticseatch is version 7.17. However no matter what I try the integration seems to be faulty. The Dashboards wont work and the Metrics are not mapped or interpreted correctly so that the Metrics Explorer will just show nothing.

This is my metricbeat.yml:

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

metricbeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

metricbeat.modules:
  - module: docker
    metricsets:
      - "container"
      - "cpu"
      - "diskio"
      - "healthcheck"
      - "info"
      - "image"
      - "memory"
      - "network"
    hosts: [ "unix:///var/run/docker.sock" ]
    period: 10s
    enabled: true
  - module: system
    metricsets:
      - cpu             # CPU usage
      - load            # CPU load averages
      - memory          # Memory usage
      - network         # Network IO
      - process         # Per process metrics
      - process_summary # Process summary
      - uptime          # System Uptime
      - socket_summary  # Socket summary
    enabled: true
    period: 10s
    processes: [ '.*' ]

    cpu.metrics: [ "percentages","normalized_percentages" ]  
    core.metrics: [ "percentages" ]  

setup.ilm.overwrite: false
setup.ilm.check_exists: false
monitoring.enabled: false

processors:
  - add_host_metadata: ~
  - add_docker_metadata: ~
  - add_fields:
      target: ''
      fields:
        environment: "test-local"
        stack: "none"

output.elasticsearch:
  hosts: ["https://xxxx.serverurl.com:443/elasticsearch"]
  username: metricbeat
  password: securepassword

kibana:
  host: https://xxxx.serverurl.com:443
  username: metricbeat
  password: securepassword

And this is my docker compose part:

services:
  metricbeat:
      image: docker.elastic.co/beats/metricbeat:7.17.13
      user: root
      volumes:
          - ./metricbeat/data:/usr/share/metricbeat/data
          - "./metricbeat.yml:/usr/share/metricbeat/metricbeat.yml:ro"
          - "/var/run/docker.sock:/var/run/docker.sock:ro"
          - "/sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro"
          - "/proc:/hostfs/proc:ro"
          - "/:/hostfs:ro"

The metricbeat logs seem to correctly reach elasticsearch, under the Discover metricbeat index I get tons of entries containing valid information.

However the Dashboard ([Metricbeat Docker] Overview ECS shows a number of running, paused and stopped containers. All the other elements show Error: "[esaggs] > Saved field "container.name" of index pattern "metricbeat-*" is invalid for use with the "Terms" aggregation Please select a new field."

And the Metrics Inventory just has no information what so ever.
When i Go into the Metrics Explorer and select cpu total I get an empty graph.
But I do have values for that.

This is how I started the setup:

docker run -d \
  --name=metricbeat \
  --user=root \
  --volume="./metricbeat.yml:/usr/share/metricbeat/metricbeat.yml:ro" \
  --volume="/var/run/docker.sock:/var/run/docker.sock:ro" \
  --volume="/sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro" \
  --volume="/proc:/hostfs/proc:ro" \
  --volume="/:/hostfs:ro" \
  docker.elastic.co/beats/metricbeat:7.17.13 metricbeat -e \
  -E output.elasticsearch.hosts=["https://xxxx.serverurl.com:443/elasticsearch"] 

I also put the false flags from the metricbeat.yml to true for the setup.

This is how I loaded the Dashboards:

docker run docker.elastic.co/beats/metricbeat:7.17.13 setup --dashboards \
  -E setup.kibana.host=https://xxxx.serverurl.com:443 \
  -E setup.kibana.username=metricbeat \
  -E setup.kibana.password=securepasswort

I tryed with the Metricbeat 7.17.12 version, I also went ahead and removed everything related to metricbeat in the elasticsearch I could find and tryed again with the 7.17.13 version. Ran all commands I could find in the Docs but nothing worked. Furthest I got was when the Dashboard showed no errors but countless failed shards.
I am pretty helpless at this point and could really use some help.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.