Chart by cmdline?

[Wanderer]

Is there a way to chart CPU utilization by process name and cmdline?

Example:
QEMU-KVM runs per VM so if you are running multiple VM's there is a QEMU-KVM process for each. Is there a way to chart the CPU usage for each one on one chart?

qemu-kvm -name kvm384 -S -machine
qemu-kvm -name kvm291 -S -machine
qemu-kvm -name kvm295 -S -machine

[spalger (Spencer Alger)]

Absolutely, but it depends on the data you put into Elasticsearch. Can you share details about what data you're collecting, how it's stored in Elasticsearch, and what Index Patterns you have setup?

[Wanderer]

Thanks for the reply. We're rookies at this so bear with me.

Index patterns are the 3 defaults if I understand it correctly for each of the following beats - metricbeat packetbeat and filebeat. We've made very few changes to the out of the box setup. The system.yml from metricbeat is:

- module: system
  period: 10s
  metricsets:
    - cpu
    - load
    - memory
    - network
    - process
    - process_summary
    - core
    - diskio
    #- socket
  processes: ['.*']
  process.include_top_n:
    by_cpu: 5      # include top 5 processes by CPU
    by_memory: 5   # include top 5 processes by memory

- module: system
  period: 1m
  metricsets:
    - filesystem
    - fsstat
  processors:
  - drop_event.when.regexp:
      system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

- module: system
  period: 15m
  metricsets:
    - uptime

- module: system
  metricsets: [core]
  core.metrics: [percentages]

- module: system
  metricsets: [cpu]
  cpu.metrics: [normalized_percentages]

[spalger (Spencer Alger)]

Okay, great, I collected a little data using your same metricbeat configuration and was able to create a bar chart that is slit by beat.hostname, but I'm only collecting data from a single host so it doesn't look split, but this is the configuration I used:

I started by creating a metricbeat* index pattern in Management > Index Pattern, entered metricbeat* as the pattern and selecting @timestamp as the time field. Then I switched over to Visualize, started a new visualization, and choose a "Vertical Bar" chart. Then I chose the metricbeat* index pattern, and setup the following aggregations to create the visualization:

• metric: type: "Max", field: system.cpu.total.pct
• buckets:
  • x-axis -- type: "Date Histogram", field: @timestamp (so the time will be stretched across the x-axis)
  • Split Chart -- type: "Terms", field: beat.hostname (so there will be one chart per hostname)

Note that the last aggregation, the "Split Chart" bucket, uses a "terms" aggregation. This looks for the terms in the specified field for the values, so it will pull the top 5 hostnames in. Since I only have data from one host it only show that data, but when you use it you'll probably see 3 charts, and if you get more VMs it should automatically update. You can also change the number of terms it allows and how it sorts what the "top x" terms are.

[Wanderer]

Thanks Spencer. I might not have been clear in my post. Let me try once more with this example.

I'd like to see the CPU utilization of 5 VM's that are running on a host.

Example: Node A is running kvm01 - kvm05. Each kvm* runs as a process on Node A. The processes look like this from atop commandline output:

/usr/libexec/qemu-kvm -name kvm01 -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1

/usr/libexec/qemu-kvm -name kvm02 -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1

/usr/libexec/qemu-kvm -name kvm03 -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1

/usr/libexec/qemu-kvm -name kvm04 -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1

/usr/libexec/qemu-kvm -name kvm05 -S -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off -m 4096 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1

Is this what you have described?

[spalger (Spencer Alger)]

No, it’s not exactly what I described. But what I described is how to make a visualization that is split by the value of a field. I can’t write the visualization for you, but look at the data you’re getting from metricbeat in discover and try to answer “which field is unique per VM?” If it isn’t beat.hostname like in my example then it must be some other field. If there is not field that is unique per VM then there is another problem.

Does that help a little?