# How to benchmark existing cluster with multi-get (API) operation

**URL:** <https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287>\
**Category:** Elasticsearch\
**Tags:** rally\
**Created:** [March 7, 2019, 11:20am UTC](https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287 "2019-03-07T11:20:25Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Suraj\_Singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/suraj_singh/32/42181_2.png) [@Suraj\_Singh](https://discuss.elastic.co/u/Suraj_Singh)\
**Post date:** [March 7, 2019, 11:20am UTC](https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287/1 "2019-03-07T11:20:26Z")

</div>

Hi Team,

I have few already created indexes(with data) on elastic and i want to get multi get performance on existing indexes.

I see we can define this with custom tracks by creating a track.json file. What should be the content of this file if I want to run 1000 mget requests per second. Each mget request contains 20 get requests.

Any help is appreciated !!

---

<div class="post-metadata">

**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\
**Post date:** [March 8, 2019, 8:11am UTC](https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287/2 "2019-03-08T08:11:31Z")

</div>

Hello,

You can find an example of a track with a single task in the docs [here](https://esrally.readthedocs.io/en/stable/track.html#examples) also showing how to limit the throughput of the executed query.

It is strongly recommended to familiarize yourself with the basic of the [anatomy of the track](https://esrally.readthedocs.io/en/stable/track.html#anatomy-of-a-track). In your case since ES already has data you won't be interested in the `indices`, `templates` and `corpora` sections of the track.

Dimitris

---

<div class="post-metadata">

**Author:** ![Suraj\_Singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/suraj_singh/32/42181_2.png) [@Suraj\_Singh](https://discuss.elastic.co/u/Suraj_Singh)\
**Post date:** [March 17, 2019, 6:10am UTC](https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287/3 "2019-03-17T06:10:25Z")

</div>

Hi Dimitrios,

I gone through the custom runner, track anatomy documentation but still struggling to run esrally to benchmark my external es cluster.

Below are the failure logs:

2019-03-17 05:39:07,308 -not-actor-/PID:43624 esrally.rally INFO Rally version [1.0.4]  
2019-03-17 05:39:07,308 -not-actor-/PID:43624 [esrally.utils.net](http://esrally.utils.net) INFO Rally connects directly to the Internet (no proxy support).  
2019-03-17 05:39:09,846 -not-actor-/PID:43624 esrally.rally INFO Detected a working Internet connection.  
2019-03-17 05:39:09,875 -not-actor-/PID:43624 esrally.rally INFO Actor system already running locally? [False]  
2019-03-17 05:39:09,875 -not-actor-/PID:43624 esrally.actor INFO Starting actor system with system base [multiprocTCPBase] and capabilities [{'coordinator': True, 'ip': '127.0.0.1', 'Convention Address.IPv4': '127.0.0.1:1900'}].  
2019-03-17 05:39:10,138 -not-actor-/PID:43629 root INFO ++++ Actor System gen (3, 9) started, admin @ ActorAddr-(T|:1900)  
2019-03-17 05:39:15,131 -not-actor-/PID:43624 esrally.actor ERROR Could not initialize internal actor system. Terminating.  
Traceback (most recent call last):  
File "/usr/local/lib/python3.7/site-packages/esrally/actor.py", line 248, in bootstrap\_actor\_system  
capabilities=capabilities)  
File "/usr/local/lib/python3.7/site-packages/thespian/actors.py", line 638, in **init**  
systemBase, capabilities, logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/actors.py", line 676, in \_startupActorSys  
systemBase = sbc(self, logDefs=logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/multiprocTCPBase.py", line 27, in **init**  
super(ActorSystemBase, self). **init** (system, logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/multiprocCommon.py", line 85, in **init**  
super(multiprocessCommon, self). **init** (system, logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/systemBase.py", line 329, in **init**  
logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/multiprocCommon.py", line 114, in \_startAdmin  
'not a valid ActorSystem admin')  
thespian.actors.InvalidActorAddress: ActorAddr-(T|:1900) is not a valid ActorSystem admin  
2019-03-17 05:39:15,141 -not-actor-/PID:43624 esrally.rally ERROR A fatal error occurred while running subcommand [race].  
Traceback (most recent call last):  
File "/usr/local/lib/python3.7/site-packages/esrally/rally.py", line 475, in dispatch\_sub\_command  
race(cfg)  
File "/usr/local/lib/python3.7/site-packages/esrally/rally.py", line 404, in race  
with\_actor\_system(lambda c: racecontrol.run(c), cfg)  
File "/usr/local/lib/python3.7/site-packages/esrally/rally.py", line 412, in with\_actor\_system  
actors = actor.bootstrap\_actor\_system(try\_join=already\_running, prefer\_local\_only=not already\_running)  
File "/usr/local/lib/python3.7/site-packages/esrally/actor.py", line 248, in bootstrap\_actor\_system  
capabilities=capabilities)  
File "/usr/local/lib/python3.7/site-packages/thespian/actors.py", line 638, in **init**  
systemBase, capabilities, logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/actors.py", line 676, in \_startupActorSys  
systemBase = sbc(self, logDefs=logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/multiprocTCPBase.py", line 27, in **init**  
super(ActorSystemBase, self). **init** (system, logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/multiprocCommon.py", line 85, in **init**  
super(multiprocessCommon, self). **init** (system, logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/systemBase.py", line 329, in **init**  
logDefs)  
File "/usr/local/lib/python3.7/site-packages/thespian/system/multiprocCommon.py", line 114, in \_startAdmin  
'not a valid ActorSystem admin')  
thespian.actors.InvalidActorAddress: ActorAddr-(T|:1900) is not a valid ActorSystem admin

---

<div class="post-metadata">

**Author:** ![Suraj\_Singh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/suraj_singh/32/42181_2.png) [@Suraj\_Singh](https://discuss.elastic.co/u/Suraj_Singh)\
**Post date:** [March 17, 2019, 9:07am UTC](https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287/4 "2019-03-17T09:07:38Z")

</div>

I am using below command  
esrally --pipeline=benchmark-only --track-path=/home/ec2-user/.rally/mytrack --target-hosts="myhosts.json"

It fails with below error message:  
[ERROR] Cannot race. Error in race control (ConnectionError((\<urllib3.connection.HTTPConnection object at 0x7ff1a1c7f2b0\>, 'Connection to 10.212.39.46 timed out. (connect timeout=60)')) caused by: ConnectTimeoutError((\<urllib3.connection.HTTPConnection object at 0x7ff1a1c7f2b0\>, 'Connection to 10.212.39.46 timed out. (connect timeout=60)')))

---

<div class="post-metadata">

**Author:** ![dliappis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dliappis/32/56174_2.png) [@dliappis](https://discuss.elastic.co/u/dliappis)\
**Post date:** [March 19, 2019, 11:07am UTC](https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287/5 "2019-03-19T11:07:15Z")

</div>

Hello,

Let's take this one step at a time.

Can you paste the content of `myhosts.json`?

Assuming Elasticsearch is using the default port `9200`, can you try, from the host where Rally is running, to verify connectivity against Elasticsearch by trying something like `nc -v 10.212.39.46:9200` (or whatever the port of ES is?)

Dimitris

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 16, 2019, 11:07am UTC](https://discuss.elastic.co/t/how-to-benchmark-existing-cluster-with-multi-get-api-operation/171287/6 "2019-04-16T11:07:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
