What's the frequnce for Rally send bulk requests

Elastic Stack Elasticsearch
1

So far as I know, Rally using async HTTP request to bulk index docs. And the implementation is in driver.py
I could perform similar requests using REST client. Consedering that asynchronised request does not wait response from ES server and REST client could send another requet. So, this cause a problem that ES server could not handle so many index requests in a quite short period.

I'm wondering to know, how often does Rally send requests to ES server? How does it calculate the period.

2

Hi @xihuanbanku,

Rally issues requests synchronously, so it does wait for a response. We have two benchmarking modes in fact:

  • Something that I'd call a "throughput benchmark": Rally issues requests as fast as it can (in a blocking fashion, not async!). This mode is usually used for bulk indexing.
  • Throughput-throttled: You can also define a target throughput and Rally aims to achieve that throughput. If Elasticsearch responds faster than that, then Rally will wait. If Elasticsearch takes longer than that, then Rally will issue the next request immediately after. So it will try to match the target throughput as close as it can. In fact, if Rally could not reach the target throughput, you will (a) see high latency numbers (much higher than the service time) and (b) this is a sign that your configuration (hardware + ES config) is at or over peak capacity and you should reduce target throughput.

Daniel

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.