Rally and Httperf

[dliappis (Dimitrios Liappis)]

Hi Eric,

@jlwysf

Regarding my second question, if "client" is the concurrent session you referred to, how many thread are related in one client, how can I know? Does one client is one process? If we only have one client , is that mean we only execute one concurrent session?

Rally uses the Thespian Actor System which, in turn, is multi-process based. My understanding is that each client corresponds to one Python3 process. Actors themselves are single threaded. See also this explanation. For the record, the instrumentation of all of this is done inside driver.py.

@jlwysf

According to you answer, I haven't seen any index called "rally-*/metrics/" is that because my existing cluster is not provisioned by esrally or they are reside in somewhere else?

Rally stores its metrics in a dedicated Elasticsearch cluster, which is highly recommended anyway so that you can execute queries, use Kibana etc. This is described in the docs here and there's information how to set it up in the advanced configuration section.

@jlwysf

According to this answer I have another question regarding how can I know how many indexing thread I have, does indexing thread is python es client you are talking about earlier?

Indexing threads refer to Elasticsearch's thread pool for indexing (in Java). As per the Elasticsearch docs:

Thread pool type is fixed with a size of # of available processors, queue_size of 200. The maximum size for this pool is 1 + # of available processors.

@jlwysf

Basically I try to run esrally against two existing cluster separately, but I only have one coordinate node, whenever I try to run parallel, simply run "pipeline benchmark only" two times with different target hosts, I got "race run error", I'm wondering is there anyway can solve this problem?

AFAIK you can't use the same Rally coordinator against two different clusters in parallel. It's better that you benchmark your clusters separately and in an isolated manner, also so that the testing results are credible and results from one cluster are not affected by testing the other cluster and vice versa.

Regards,
Dimitris