Metricbeat not getting diskio

Elastic Stack Beats
1

where can I see diskio for process and network traffic use by process?

I did load default template, running 7.9.1
here is config file

- module: system
  period: 1m
  metricsets:
    - cpu
    - load
    - memory
    - network
    - process
    - process_summary
    - socket_summary
    - entropy
    - core
    - diskio
    #- socket
    - service
    - users
  process.include_top_n:
    by_cpu: 10      # include top 5 processes by CPU
    by_memory: 10   # include top 5 processes by memory

- module: system
  period: 5m
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib|snap)($|/)'

- module: system
  period: 15m
  metricsets:
    - uptime

Here is output of dd command that ran.

"_source": {
    "@timestamp": "2020-10-09T18:21:54.276Z",
    "service": {
      "type": "system"
    },
    "process": {
      "executable": "/usr/bin/dd",
      "args": [
        "dd",
        "if=/dev/sdb1",
        "of=/dev/null"
      ],
      "name": "dd",
      "pid": 36666,
      "ppid": 33543,
      "pgid": 36666,
      "working_directory": "/etc/metricbeat"
    },
    "event": {
      "duration": 310303583,
      "dataset": "system.process",
      "module": "system"
    },
    "metricset": {
      "name": "process",
      "period": 10000
    },
    "ecs": {
      "version": "1.5.0"
    },
    "system": {
      "process": {
        "fd": {
          "open": 3,
          "limit": {
            "soft": 1024,
            "hard": 4096
          }
        },
        "state": "running",
        "memory": {
          "size": 110604288,
          "rss": {
            "bytes": 753664,
            "pct": 0
          },
          "share": 675840
        },
        "cmdline": "dd if=/dev/sdb1 of=/dev/null",
        "cpu": {
          "start_time": "2020-10-09T18:21:48.000Z",
          "total": {
            "pct": 0,
            "norm": {
              "pct": 0
            },
            "value": 0
          }
        }
      }
    },
    "user": {
      "name": "root"
    },
    "agent": {
      "version": "7.9.1",
      "hostname": "elktst02",
      "ephemeral_id": "9c14dd0c-a5de-4627-8514-a0d68220dc3b",
      "id": "17eb491f-8a11-4e66-96e4-47cc8a86203c",
      "name": "houelktst02",
      "type": "metricbeat"
    },
    "host": {
      "name": "elktst02"
    }
  },
  "fields": {
    "system.process.cpu.start_time": [
      "2020-10-09T18:21:48.000Z"
    ],
    "@timestamp": [
      "2020-10-09T18:21:54.276Z"
    ]
  },
2

Hey @elasticforme,

I think there are currently no diskio metrics per process available. Only per disk. Imho it would be a nice addition.

Grtz

3

you mean currently metricbeat can only collect cpu,memory,fd for each process. nothing else?

4

but this document says it should be there
system.process.cgroup.blkio.total.bytes

https://www.elastic.co/guide/en/beats/metricbeat/7.9/metricbeat-metricset-system-process.html

5

Well I'd be suprised it's currently possible. Never seen those metrics in my data anyway. Also checking https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-metricset-system-process.html I can't seem to find any info about it.

6

same link it is there

"system": {
        "process": {
            "cgroup": {
                "blkio": {
                    "id": "docker.service",
                    "path": "/system.slice/docker.service",
                    "total": {
                        "bytes": 844576104448,
                        "ios": 54869430
                    }
                },
7

Did you try set:

process.cgroups.enabled: true

8

ok just added that restarted metricbeat. restarted same process. but no I still didn't got that cgroup. infact all the cgroup are empty.

9

Im not sure, but these cgroup metrics might be related to container metrics, I suggest we wait for someone from Elastic to confirm. Sry I cant help you further.

10

you might be right. just found this and it is close as well without any fix

https://github.com/elastic/beats/issues/7461

Hopefully someone from ELK can confirm this.

11

+1'ed it, maybe the enhancement request can get reopened.

12

Added a comment in the GH issue to question the closure, hopefully someone of Elastic sees the added value.

1 Like
13

I also did +1 on that issue

1 Like
14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.