Host Metrics not captured for centos7

Elastic Stack Beats
1

Hi,

I'm running metricbeat in a docker container. I'm unable to see the host's metrics in the dashboard (e.g. cpu usage, memory, etc.).

Below is what i have in my docker compose file in terms of volume mounts and flags:

 metricbeat:
    image: metricbeat
    user: root
    restart: on-failure
    networks:
      - mynet
    volumes:
      - /proc:/hostfs/proc:ro
      - /sys/fs/cgroup:/hostfs/sys/fs/cgroup:ro
      - /:/hostfs:ro
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - ELASTICSEARCH_HOST=http://elasticsearch:9200
      - KIBANA_HOST=https://kibana:5001
    command: ["--strict.perms=false", "-system.hostfs=/hostfs", "-"]
    depends_on:
      - elasticsearch

And here are the contents of my metricbeat.yml:

#-------------------------------- Autodiscovery -------------------------------
metricbeat.autodiscover:
  providers:
    - type: docker
      templates:
        - condition.contains:
            docker.container.image: redis
          config:
            - module: redis
              metricsets: ["info", "keyspace"]
              hosts: "${data.host}:${data.port}"

metricbeat.modules:
#------------------------------- System Module -------------------------------
- module: system
  metricsets: ["cpu", "load", "memory", "network", "process", "process_summary", "core", "diskio", "socket"]
  processes: ['.*']
  process.include_top_n:
    by_cpu: 5
    by_memory: 5
  period: 10s
  cpu.metrics:  ["percentages"]
  core.metrics: ["percentages"]

- module: system
  period: 1m
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

- module: system
  period: 15m
  metricsets:
    - uptime

#------------------------------- Docker Module -------------------------------
- module: docker
  metricsets: ["container", "cpu", "diskio", "healthcheck", "info", "memory", "network"]
  hosts: ["unix:///var/run/docker.sock"]
  period: 10s

#------------------------------- Processors -------------------------------
processors:
- add_docker_metadata: ~
- add_cloud_metadata: ~
- add_locale: ~

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  hosts: ["http://elasticsearch:9200"]

xpack.monitoring:
  enabled: true
  elasticsearch:

#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index.
# Loading the dashboards is disabled by default and can be enabled either by setting the options here, or by using the `-setup` CLI flag.
setup.dashboards:
  enabled: true

setup.dashboards.retry.enabled: true

setup.kibana:
  host: "https://kibana:5601"
  protocol: "https"
  ssl.enabled: true
  ssl.certificate_authorities: ["/usr/share/metricbeat/certs/ca.crt"]
  ssl.certificate: "/usr/share/metricbeat/certs/kibana.crt"
  ssl.key: "/usr/share/metricbeat/certs/kibana.key"

Does anything look wrong to you? Am I missing something? Your help is much appreciated.

Thanks,
Jas

2

Hi @Jas_Ahluwalia :slight_smile:

Frankly, I don't see any errors on your config files. Can you paste some log outputs?

3

Hi,

Here is a link to the logfile for metricbeat:

https://file.io/wLVFxc

Thanks,
jas

4

404 not found :upside_down_face:

5

Hmm....let's try this one:

I didn't see much in there, i think there is something about elasticsearch not being up, but eventually it comes up and everything should be fine. But maybe your eyes will catch something.

Thanks,
Jas

6

You commented that you were using docker compose and the config of Elasticsearch seems quite barebones. I'm wondering if it has something in relation with launching it in cluster mode? Maybe try with the "single node" launch docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:6.4.2 as described here:

https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html#docker-cli-run

I don't think this is the problem but meanwhile I'm gonna ask some partner too because I can't see anything relevant in the logs either

7

Appreciate it. And yeah, single node didn't really have an effect.

Thanks,
Jas

8

Hi Mario,

I think I was able to figure out something. I had removed the system dashboard from my container, and I believe the host dashboard was going to a hard coded host id for the host dashboard that didn't exist. When i revert back to the original dashboards, things seem to look better.

However, when I chose "last 15 mins," everything alwasy says zero (processor, mem usage). If I say last 30 mins, i start to get some data.

Can I get some guidance on how to get the settings in my metricbeat.yml and "last 15 mins" to line up better? is there a notion of "current live view?"

Thanks,
Jas

9

Forgive me for the delay @Jas_Ahluwalia

@exekias can you help us with this issue, please?

10

No worries. I'm still having the same issue with time interval at 'last 15 mins.'

Thanks,
Jas

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.