Disk IO - How to visualize Max Disk Throughput

[erikg]

Hello is there a way to visualize the Disk Throughput using Lens?

I want to do a line graph but it seems like its not possible to do it with Lens.

Here's the fields:
system.diskio.read.bytes

The total number of bytes read successfully. On Linux this is the number of sectors read multiplied by an assumed sector size of 512.

type: long

format: bytes

system.diskio.write.bytes

The total number of bytes written successfully. On Linux this is the number of sectors written multiplied by an assumed sector size of 512.

type: long

format: bytes

system.diskio.read.time

The total number of milliseconds spent by all reads.

type: long

system.diskio.write.time

The total number of milliseconds spent by all writes.

type: long

[Jesselastic]

I'm surprised this didn't get hit by the 28 days later zombies. I am currently trying to work on this too, but I am thinking the system.diskio.io.time would cover both read and write time, but so far not getting good results.

[stephenb (Stephen Brown)]

Hi @Jesselastic

What version Elasticsearch?
What / how are you collecting data? elastic Agent Metricbeat etc?
Which exact fields?

and

What exactly are you trying to calculate?

Example with Elastic Agent

Quick Query Bar Filter

data_stream.dataset : "system.diskio" and agent.name : "stephenb-metrics-test-es1" and metricset.name : "diskio"

X - Axis Date Histogram

Y-Axis : counter_rate(max(system.diskio.io.time))

IMPORTANT: Breakdown by system.diskio.io.name to 20 or so because there is more than 1 disk name... if you do not do this your results will .... not... work....

You can collapse by SUM if you want the total or none if you want to see each disk... ... This is know as a series aggregation ... This is in the breakdown ... now this will sum up all the individual counter rates by system.diskio.name the different for the total sum.

Now you get a single like that is the SUM of all the indvidual disk ios

Here is the Mappings for Elastic Agent ...

So what exactly do you want?

       "system": {
          "properties": {
            "diskio": {
              "properties": {
                "io": {
                  "properties": {
                    "ops": {
                      "type": "long"
                    },
                    "time": {
                      "type": "long",
                      "time_series_metric": "counter"
                    }
                  }
                },
                "name": {
                  "type": "keyword",
                  "time_series_dimension": true
                },
                "read": {
                  "properties": {
                    "bytes": {
                      "type": "long",
                      "meta": {
                        "unit": "byte"
                      },
                      "time_series_metric": "counter"
                    },
                    "count": {
                      "type": "long",
                      "time_series_metric": "counter"
                    },
                    "time": {
                      "type": "long",
                      "time_series_metric": "counter"
                    }
                  }
                },
                "serial_number": {
                  "type": "keyword",
                  "ignore_above": 1024
                },
                "write": {
                  "properties": {
                    "bytes": {
                      "type": "long",
                      "meta": {
                        "unit": "byte"
                      },
                      "time_series_metric": "counter"
                    },
                    "count": {
                      "type": "long",
                      "time_series_metric": "counter"
                    },
                    "time": {
                      "type": "long",
                      "time_series_metric": "counter"
                    }
                  }
                }
              }
            }
          }
        }
      }
    }

Here is is a Lens

X Axis :