Visualization comparing data terms in the same row

Elastic Stack Kibana
#1

I would like to create a bar graph visualization that compares 2 data elements. One is an estimate of how long a process should run in seconds and the other is the actual run time in milliseconds. It should group and count the estimates and display the number of items that exceeded the estimate and by how much and the number that ran in a shorter time and by how much.

To clarify, there would be a bar for each estimated time (or perhaps the top 25). For each bar, I want to see how many were over the estimate, and how many were under, rounded to the nearest half second. These would be stacked bars so I can see how many were 1 second long, how many were 2 seconds long, how many were 1 second short, etc.

#2

I'm having trouble understanding your description. It would help if you provided a couple of sample documents in Elasticsearch of the data you're working with, a mock visualization you're trying to create (even if it's just a hand-drawing), as well as a screenshot of something you've already tried in Kibana, even if it falls short.

#3

Below is example data.

kibana-example-data.PNG1893×935 80.9 KB

kibana-example-data2.PNG1547×841 70.7 KB

The columns I want to compare are duration, which is in milliseconds, and time_allocated which is in seconds. For this data, I want to see a bar for time_allocated = 2 and one for 3. I want the bars divided into segments representing durations that exceed the allocation on the positive side and durations that are less than the allocation on the negative side. Since duration is in milliseconds, the values need to be divided by1000 before the comparison with time_allocated.

This is the only thing I've come up with so far and it really doesn't come close to what I want.

kibana-example-visualization.PNG1174×445 27.2 KB

The bars should be centered on zero with segments that are the difference between duration/1000 - time_allocated.

The goal of this is to give me a quick visualization for how well my code is estimating job duration.

#4

Have you tried creating a scripted field to represent "duration/1000 - time_allocated" and trying to visualize that?

#5

No, I hadn't run across that feature. I'll give it a try. Thanks.

#6

This is what I ended up with for a scripted field:
ceil(doc['time_allocated'].value - (doc['duration'].value / 1000))

This gives me a negative integer when the process runs longer than the estimate and a positive integer when I over estimate. I used it to create this:

allocated-vs-duration.PNG788×344 14.5 KB

This graph shows, in percentiles, that I am allocating more than I need to most of the time.

Thanks again for the help.

1 Like
#7

That looks really cool! Thanks for sharing :slight_smile:

#8