I am benchmarking ES cluster using esrally. I have created custom tracks using ES queries only search operation not indexing. There is a consistent individual query performance gap(considering 100th percentile service time and Max Throughput) between c5 and i3 instances but the difference in the overall track execution time is 1-2 minutes.
What are the metrics I need to consider for benchmarking? Please suggest.
Hello,
Thanks for your interest in Rally.
First of all, I'd recommend to take a look at this presentation for an overview of common pitfalls in benchmarks.
One of the things you'll find there is the notion of warm up time, so you should ensure that your the JVM, caches etc. have all warmed up sufficiently before you consider the measurement samples report worthy. (See warmup-iterations and warmup-time-period in the docs).
You should also monitor the usage on your instance (e.g. using the USE methodology); this is also discussed in the presentation linked above. c5 and i3 are vastly different in CPU, Disk and network capabilities and e.g. things can differ depending on how disk setup's done.
Thanks for the response.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.