Today we're proud to announce Rally 0.4.0. Although not much changed from the outside, Rally 0.4.0 is basically a rewrite of one of its core components: the load generator.
How to...?
- ... upgrade:
pip3 install --upgrade esrally - ... install:
pip3 install esrally
Note: Depending on your system setup you probably need to prepend these commands with sudo.
Please follow the installation instructions for a first time install though.
Changes
Contrary to the old thread-based approach, this approach uses now multiple processes and has shown a better peak (and mean) throughput in driver stress tests, can execute search requests in parallel (although we will need to improve that a bit in an upcoming maintenance version).
We also got rid of a few conceptual problems of the old driver:
- Rally 0.4.0 does not suffer anymore from a problem called "coordinated omission" that many - even very popular - load generators like JMeter have. If you want to know more about this phenomenon watch the talk How NOT to measure latency by Gil Tene on Youtube.
- Rally also allows throughput throttling (which is basically a consequence of eliminating coordinated omission).
- We show service time and latency separately in the results (see their definition in the docs).
- You can specify the warmup time period for bulk index operations in the track. Previously, this was hard-coded in Rally.
Besides, Rally as well as the official Rally tracks dropped support for Elasticsearch 5.0.0-alpha1 and 5.0.0-alpha2, all other versions of Elasticsearch should still work.
What's next?
This is a significant step forward for Rally. We plan to smooth out a few things in the coming 0.4 releases:
- Improve feedback messages: As of Rally 0.4.0 the fine-grained feedback is gone and Rally will update the percentage count only after each phase. This means you'll see 0% progress for quite some time but we'll plan to reintroduce the old feature soon again.
- Add a ramp-down phase: Currently we only have a warmup phase and a measurement phase in Rally. For multiple reasons it makes sense to also include a ramp-down phase in benchmarks.
- We'll also attempt to bring back CPU statistics per benchmark phase although this could get a bit hairy.
After that the new big goal on the roadmap is finally support for benchmarks of arbitrarily large clusters in version 0.5.0. Internally, we're now equipped to do that with 0.4.0.
If you have questions or feedback, please just post in the Rally forum. File any issues in the Rally Github repository.