Some questions on system.diskio latency metrics

Elastic Observability Metrics
1

Hello, I am trying to get a better understanding of the disk io metrics below.
Included are the definitions for each per the Elastic documentation online.

system.diskio.read.time - total number of milliseconds spent by all reads
system.diskio.write.time - total number of milliseconds spent by all writes
system.diskio.write.time - The total number of of milliseconds spent doing I/Os.
system.diskio.read.bytes - The total number of bytes read successfully
system.diskio.write.bytes - The total number of bytes write successfully
system.diskio.read.count - The total number of reads completed successfully.
system.diskio.write.count - The total number of writes completed successfully.

Q1: In the definitions of system.diskio.read/write.time, the total number of milliseconds spent by all reads/writes.I found that the total time is very large, far greater than the sampling period I set. SO, what time does this total time consist of？
Q2：Similar to Q1，what bytes does this total bytes consist of？
Q3: Does system.diskio.write.read/count record all read/write operations in the event.duration time？

2

These metrics are counters. This is how they recorded on OSs

That means they a monotonically increasing numbers.

Counter
A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero on restart. For example, you can use a counter to represent the number of requests served, tasks completed, or errors.

So to calculate the write time per bucket / time range

write.time(1) - write.time(0)

That is the total ms spent on writes during that time span.

Look closely you probably will want to look at them by

system.filesystem.device_name

And to calculate the like read/write MB/s

So to visualize them in Lens use the function counter rate.

So to calculate a rate it is a derivative....

(write.bytes(1) - write.bytes(0))/time bucket span

Look closely you probably will want to look at them by

system.filesystem.device_name
Can also look at the OOTB dashboard and visualizations to see..

3

Thank you. But I still have a question. Displayed in the metricbeat document, disk contains disk IO metrics collected from the operating system. So,does Linux have commands that can obtain similar data？

4

Well that is a bit outside elasticsearch topics....

You can look it up on Google

You can also look at the metricbeat code if you like