Hi,
I am a Rally newbie and having a go at creating some custom tracks and cars. My OS is Ubuntu Linux 18.04.1 64-bit. Rally version is 1.4.1. I am attempting to align my testing with ES v6.6.0 for reasons I won't go into, although I realise it's a year or two old.
So far, I am able to list my new cars.
Summary
spayne:~ $ esrally list cars --team-repository ~/elasticsearch-src/teams
____ ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
Available cars:
Name Type Description
----------------------- ------ --------------------------------------------------------
16gheap car Sets the Java heap to 16GB
1gheap car Sets the Java heap to 1GB
24gheap car Sets the Java heap to 24GB
2gheap car Sets the Java heap to 2GB
4gheap car Sets the Java heap to 4GB
8gheap car Sets the Java heap to 8GB
defaults car Sets the Java heap to 1GB
java-bridge-debug car SOME DESCRIPTION
java-no-debug car SOME DESCRIPTION
java-standard-debug car SOME DESCRIPTION
basic-license mixin Basic License
debug-non-safepoints mixin More accurate CPU profiles
ea mixin Enables Java assertions
fp mixin Preserves frame pointers
g1gc mixin Enables the G1 garbage collector
trial-license mixin Trial License
unpooled mixin Enables Netty's unpooled allocator
x-pack-ml mixin X-Pack Machine Learning
x-pack-monitoring-http mixin X-Pack Monitoring (HTTP exporter)
x-pack-monitoring-local mixin X-Pack Monitoring (local exporter)
x-pack-security mixin X-Pack Security
-------------------------------
[INFO] SUCCESS (took 0 seconds)
-------------------------------
and tracks:
Summary
spayne:~ $ esrally list tracks --track-path=~/elasticsearch-src/custom-tracks
____ ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
Available tracks:
Name Description Documents Compressed Size Uncompressed Size
------------- ---------------------------- ----------- ----------------- -------------------
custom-tracks Tutorial benchmark for Rally 11,658,903 N/A 1.4 GB
-------------------------------
[INFO] SUCCESS (took 0 seconds)
-------------------------------
However when I try to run the benchmark I get an error.
When I run this command from the console:
esrally --car=java-no-debug --distribution-version=6.6.0 --track-path=~/elasticsearch-src/custom-tracks --team-path=~/elasticsearch-src/team
it gives an error:
Summary
____ ____
/ __ \____ _/ / /_ __
/ /_/ / __ `/ / / / / /
/ _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
/____/
[INFO] Preparing for race ...
[ERROR] Cannot race. ('Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct.', None)
Traceback (most recent call last):
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 355, in fetch
net.download(download_url, distribution_path, progress_indicator=progress)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 152, in download
expected_size_in_bytes = download_http(url, tmp_data_set_path, expected_size_in_bytes, progress_indicator)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 116, in download_http
raise urllib.error.HTTPError(url, r.status, "", None, None)
urllib.error.HTTPError: HTTP Error 404:
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 565, in receiveMsg_StartNodes
self.mechanic.start_engine()
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 676, in start_engine
binaries = self.supply()
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 218, in __call__
supplier.fetch()
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 361, in fetch
"version [%s] is correct." % (download_url, self.version))
esrally.exceptions.SystemSetupError: ('Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct.', None)
From checking the available past releases, there is one similar:
Available:
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0.tar.gz
My stuff wants:
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz
Is it possible to pass a modified distribution-version param to locate the correct remote release version and fix this?
Alternatively (even better), is it possible for me to run this with the same ES distro that I had already downloaded to my machine to use "for real" rather than pull another one down just to run the rally? I'd prefer that to using a separate version as it's the one I am focused on.
Or is there something broken on the download site?
Please advise, thanks.
Simon
P.S. my rally.ini file:
[meta]
config.version = 17
[system]
env.name = local
[node]
root.dir = /home/spayne/.rally/benchmarks
src.root.dir = /home/spayne/elasticsearch-src
[source]
remote.repo.url = https://github.com/elastic/elasticsearch.git
elasticsearch.src.subdir = elasticsearch
[benchmarks]
local.dataset.cache = /home/spayne/.rally/benchmarks/data
[reporting]
datastore.type = in-memory
datastore.host =
datastore.port =
datastore.secure = False
datastore.user =
datastore.password =
[tracks]
default.url = https://github.com/elastic/rally-tracks
[teams]
default.url = https://github.com/elastic/rally-teams
[defaults]
preserve_benchmark_candidate = False
[distributions]
release.cache = true
and rally.log:
Summary
(truncated to last few lines)
2020-02-27 11:09:14,926 ActorAddr-(T|:38069)/PID:5198 esrally.metrics INFO Creating file race store
2020-02-27 11:09:14,926 ActorAddr-(T|:38069)/PID:5198 esrally.mechanic.mechanic WARNING Cannot store system metrics: ('No race with race id [b50d6f7e-a699-4d8d-a894-4dd27713b510]', None).
2020-02-27 11:09:14,926 ActorAddr-(T|:38069)/PID:5198 esrally.metrics INFO Closing metrics store.
2020-02-27 11:09:14,928 ActorAddr-(T|:33407)/PID:5196 esrally.actor ERROR Child actor exited with [ChildActorExited:ActorAddr-(T|:41233)] while in status [starting].
2020-02-27 11:09:14,943 ActorAddr-(T|:34751)/PID:5195 esrally.actor INFO BenchmarkActor received unknown message [ChildActorExited:ActorAddr-(T|:33407)] (ignoring).
2020-02-27 11:09:17,926 -not-actor-/PID:5186 esrally.rally INFO Attempting to shutdown internal actor system.
2020-02-27 11:09:17,928 -not-actor-/PID:5194 root INFO ActorSystem Logging Shutdown
2020-02-27 11:09:17,949 -not-actor-/PID:5193 root INFO ---- Actor System shutdown
2020-02-27 11:09:17,950 -not-actor-/PID:5186 esrally.rally INFO Actor system is still running. Waiting...
2020-02-27 11:09:18,952 -not-actor-/PID:5186 esrally.rally INFO Shutdown completed.
2020-02-27 11:09:18,953 -not-actor-/PID:5186 esrally.rally ERROR Cannot run subcommand [race].
Traceback (most recent call last):
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/rally.py", line 714, in dispatch_sub_command
race(cfg)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/rally.py", line 624, in race
with_actor_system(racecontrol.run, cfg)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/rally.py", line 652, in with_actor_system
runnable(cfg)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 365, in run
raise e
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 362, in run
pipeline(cfg)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 79, in __call__
self.target(cfg)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 290, in from_distribution
return race(cfg, distribution=True)
File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 250, in race
raise exceptions.RallyError(result.message, result.cause)
esrally.exceptions.RallyError: (Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct., 'Traceback (most recent call last):\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 360, in fetch\n net.download(download_url, distribution_path, progress_indicator=progress)\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 152, in download\n expected_size_in_bytes = download_http(url, tmp_data_set_path, expected_size_in_bytes, progress_indicator)\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 116, in download_http\n raise urllib.error.HTTPError(url, r.status, "", None, None)\nurllib.error.HTTPError: HTTP Error 404: \n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 565, in receiveMsg_StartNodes\n self.mechanic.start_engine()\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 676, in start_engine\n binaries = self.supply()\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 218, in __call__\n supplier.fetch()\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 366, in fetch\n "version [%s] is correct." % (download_url, self.version))\nesrally.exceptions.SystemSetupError: (\'Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct.\', None)\n')