Esrally loaddriver configurations

Elastic Stack Elasticsearch
1

Hi All,

I am trying to setup esrally in k8s, As per the Tips and Tricks — Rally 2.2.1 documentation it is good approach or solution to distribute the load to loaddrivers. what would be the better configurations for loaddrivers(like cpu,ram). kindly suggest.

2

Since you intend to use Rally in k8s, I assume you'll be using the Rally Docker image. As mentioned in the docs the Rally Docker image doesn't (currently at least) support distributing the load driver, so it'd be best to ensure that wherever Rally ends up running, limited resources on the load driver are not going to be a cause for bottlenecking your benchmark.

Depending on which track you use you may need to ensure there is ample amount of memory available; the standard tracks in GitHub - elastic/rally-tracks: Track specifications for the Elasticsearch benchmarking tool Rally are of static nature and therefore most challenges don't require a whole lot of RAM, with the exception of challenges that simulate update workloads in which case more data needs to be held in RAM. Other tracks, like GitHub - elastic/rally-eventdata-track: Rally track for simulating event-based data use-cases, generate data -- for indexing and querying -- dynamically and therefore require more CPU and Memory.

More concretely I'd check:

  1. That the Rally container always ends up running on the same underlying host to reduce run to run variation (this is of course more pertinent to the target Elasticsearch itself, but it's good to ensure stable performance on the load driver too) and that the container doesn't end up running on a host that is severely overloaded.

  2. Container and underlying host metrics while the benchmark is running to ensure that the benchmark isn't affected by CPU/Disk IO/Network/Memory bottlenecks.

3

Thank you for the information.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.