Rally throughput metric as average instead of percentile

Elastic Stack Elasticsearch
1

Hi,

I have been using the rally to benchmark the setup. The throughput latency and service time are in percentile. Is there any to way to get the throughput metrics in average response time (latency + service) rather than percentile?

I know the reason why rally throughput metrics are in percentile. But I stiill need to know the average response time.

2

Hi,

the average is really problematic because it is very sensitive to outliers. In Rally we usually use the median instead which is more robust and the 50th percentile is just another name for the median so maybe this already helps?

If you setup a dedicated metrics store for Rally then Rally will store all raw request metrics in its metrics store. If you really want to calculate the average then you can use those raw metrics records to calculate it.

Daniel

3

Thank u @danielmitterdorfer :slight_smile: for swift response.

Is there any plugin to analyse rally metrics and represent it in visual graph to check against its previous run? Currently, we have to do this manually.

4

I found the word "latency" in metric report sounds confusing and the definition of it. The word "latency" is not intuitive. In rally, "latency" means "wait period + service time", but its not what you think when u hear it for the first time. I was thinking "latency" has to do only with wait period of the request and "service time" which gives actual time spent to serve the request. Maybe, better naming would avoid this confusion. The definition of latency is well documented though, it would save someone time. :slight_smile:

5

Hi,

I agree that the problem is that different tools use different terms and some use the terms latency, response time or service time very loosely. However, I disagree that these values are not very clearly documented in Rally. It's quite the contrary and we explicitly define those terms in our documentation:

We also have a dedicated FAQ item about the differences.

However, we're always open for improvements and happy to receive pull requests. :slight_smile:

Daniel

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.