Obviously the results from Rally are relative and depend on the cluster configuration. That said, it would be useful if there was a reference guide to explain what would be a good or a bad result for each of the scores. Does such a thing exist?
there is not so much in the official docs about that but I gave a talk around this topic at Javazone last year (slides). They usually record the talks but there was a technical problem with mine. However another video with a very similar version of this talk is available.
I think it is crucial to understand the relationship between service time, latency and throughput and all these terms are explained in the talk (around 35 minutes in the talk; I overhauled slides quite a bit from the video version to make this even clearer).
I plan to improve the documentation on this front anyway but I am also happy to accept PRs.
If something is not clear after you've watched the talk or looked at the slides (hint: there is bonus material at the end of the slides), please just ask questions here or even raise a doc issue in the Rally Github repo.
I watched your talk -- good presentation, and informative. We'll reapproach our benchmarking with several improvements to our process.
That said I think it would be useful to have documentation that talked about the relationships between the scores in a healthy configuration. Might be a stretch, but perhaps Rally itself could identify "green", "yellow" or "red" findings, just based on the relative differences rather than the whole numbers.
You might also offer a reference benchmark on a known cluster config. E.g. "results from geodata benchmarks with 1 dedicated master of m4.large & 2 data nodes of r4.large on a well configured(*) cluster". That'd help give some confidence about a given configuration.
Nice work. Thanks!
glad to hear that the talk has helped you.
Interesting idea; I need to ponder that.
We have something like this in the making but I fear it will take some time.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.