Esrally warmup explaination

[fentom (Tom)]

Hi,
I'm new to Elasticsearch and ESRally. I want to run some benchmarks but I don't understand how the warmup-iterations parameter for tracks works. Documentation says "Number of iterations that each client should execute to warmup the benchmark candidate. Warmup iterations will not show up in the measurement results."
But I dont understand number of iterations of what?

For example if I have an bulk operation with warmup-iterations=100 what exactly does this mean?
Does it mean that the bulk starts instead of measuring from interation 0, it starts measuring from iteration 101?

[Quentin_Pradet]

Hi! Good questions.

@fentom

But I dont understand number of iterations of what?

For example if I have an bulk operation with warmup-iterations=100 what exactly does this mean?

It's the number of iterations of your operation. If your operation is a search query, then Rally will execute this search query 100 times before actually taking the measurements into account.

@fentom

Does it mean that the bulk starts instead of measuring from interation 0, it starts measuring from iteration 101?

Yes!

You can find more details in the following blog post: Seven Tips for Better Elasticsearch Benchmarks | Elastic Blog. The example is about indexing throughput, where warmup is configured using warmup-time-period, but the idea is the same. Here's another example, where warmup is probably a bit too long:

You can see that the query time stabilizes quite early, but Rally still discards all warmup measurements, to only use the normal ones. (I was able to create this visualization in Kibana because Rally still stores warmup results in the metrics store.)

[fentom (Tom)]

Thank you for this explaination