# Cannot download ES distribution 6.6.0 as rally builds incorrect download path

**URL:** <https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207>\
**Category:** Elasticsearch\
**Tags:** rally\
**Created:** [February 27, 2020, 10:14am UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207 "2020-02-27T10:14:38Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sipayne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sipayne/32/63416_2.png) [@sipayne](https://discuss.elastic.co/u/sipayne)\
**Post date:** [February 27, 2020, 10:14am UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207/1 "2020-02-27T10:14:38Z")

</div>

Hi,

I am a Rally newbie and having a go at creating some custom tracks and cars. My OS is Ubuntu Linux 18.04.1 64-bit. Rally version is 1.4.1. I am attempting to align my testing with ES v6.6.0 for reasons I won't go into, although I realise it's a year or two old.

So far, I am able to list my new cars.

> **Summary**
>
> ```auto
> spayne:~ $ esrally list cars --team-repository ~/elasticsearch-src/teams
> 
> ________
> / __\____ _/ / /_ __
> / /_/ / __ `/ / / / / /
> / _, _/ /_/ / / / /_/ /
> /_/ |_|\ __,_/_/_/\__ , /
> / ____ /
> 
> Available cars:
> 
> Name Type Description
> ----------------------- ------ --------------------------------------------------------
> 16gheap car Sets the Java heap to 16GB
> 1gheap car Sets the Java heap to 1GB
> 24gheap car Sets the Java heap to 24GB
> 2gheap car Sets the Java heap to 2GB
> 4gheap car Sets the Java heap to 4GB
> 8gheap car Sets the Java heap to 8GB
> defaults car Sets the Java heap to 1GB
> java-bridge-debug car SOME DESCRIPTION
> java-no-debug car SOME DESCRIPTION
> java-standard-debug car SOME DESCRIPTION
> basic-license mixin Basic License
> debug-non-safepoints mixin More accurate CPU profiles
> ea mixin Enables Java assertions
> fp mixin Preserves frame pointers
> g1gc mixin Enables the G1 garbage collector
> trial-license mixin Trial License
> unpooled mixin Enables Netty's unpooled allocator
> x-pack-ml mixin X-Pack Machine Learning
> x-pack-monitoring-http mixin X-Pack Monitoring (HTTP exporter)
> x-pack-monitoring-local mixin X-Pack Monitoring (local exporter)
> x-pack-security mixin X-Pack Security
> 
> -------------------------------
> [INFO] SUCCESS (took 0 seconds)
> -------------------------------
> 
> ```

and tracks:

> **Summary**
>
> ```auto
> spayne:~ $ esrally list tracks --track-path=~/elasticsearch-src/custom-tracks
> 
> ________
> / __\____ _/ / /_ __
> / /_/ / __ `/ / / / / /
> / _, _/ /_/ / / / /_/ /
> /_/ |_|\ __,_/_/_/\__ , /
> / ____ /
> 
> Available tracks:
> 
> Name Description Documents Compressed Size Uncompressed Size
> ------------- ---------------------------- ----------- ----------------- -------------------
> custom-tracks Tutorial benchmark for Rally 11,658,903 N/A 1.4 GB
> 
> -------------------------------
> [INFO] SUCCESS (took 0 seconds)
> -------------------------------
> 
> ```

However when I try to run the benchmark I get an error.  
When I run this command from the console:

```auto
esrally --car=java-no-debug --distribution-version=6.6.0 --track-path=~/elasticsearch-src/custom-tracks --team-path=~/elasticsearch-src/team

```

it gives an error:

> **Summary**
>
> ```auto
> ________
> / __\____ _/ / /_ __
> / /_/ / __ `/ / / / / /
> / _, _/ /_/ / / / /_/ /
> /_/ |_|\ __,_/_/_/\__ , /
> / ____ /
> 
> [INFO] Preparing for race ...
> [ERROR] Cannot race. ('Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct.', None)
> Traceback (most recent call last):
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 355, in fetch
> net.download(download_url, distribution_path, progress_indicator=progress)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 152, in download
> expected_size_in_bytes = download_http(url, tmp_data_set_path, expected_size_in_bytes, progress_indicator)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 116, in download_http
> raise urllib.error.HTTPError(url, r.status, "", None, None)
> urllib.error.HTTPError: HTTP Error 404: 
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 565, in receiveMsg_StartNodes
> self.mechanic.start_engine()
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 676, in start_engine
> binaries = self.supply()
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 218, in __call__
> supplier.fetch()
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 361, in fetch
> "version [%s] is correct." % (download_url, self.version))
> esrally.exceptions.SystemSetupError: ('Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct.', None)
> 
> ```

From checking the available past releases, there is one similar:  
Available:  
[https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0.tar.gz](https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0.tar.gz)  
My stuff wants:  
[https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86\_64.tar.gz](https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz)

Is it possible to pass a modified distribution-version param to locate the correct remote release version and fix this?

Alternatively (even better), is it possible for me to run this with the same ES distro that I had already downloaded to my machine to use "for real" rather than pull another one down just to run the rally? I'd prefer that to using a separate version as it's the one I am focused on.

Or is there something broken on the download site?

Please advise, thanks.

Simon

P.S. my rally.ini file:

```auto
[meta]
config.version = 17

[system]
env.name = local

[node]
root.dir = /home/spayne/.rally/benchmarks
src.root.dir = /home/spayne/elasticsearch-src

[source]
remote.repo.url = https://github.com/elastic/elasticsearch.git
elasticsearch.src.subdir = elasticsearch

[benchmarks]
local.dataset.cache = /home/spayne/.rally/benchmarks/data

[reporting]
datastore.type = in-memory
datastore.host = 
datastore.port = 
datastore.secure = False
datastore.user = 
datastore.password = 

[tracks]
default.url = https://github.com/elastic/rally-tracks

[teams]
default.url = https://github.com/elastic/rally-teams

[defaults]
preserve_benchmark_candidate = False

[distributions]
release.cache = true

```

and rally.log:

> **Summary**
>
> ```auto
> (truncated to last few lines)
> 2020-02-27 11:09:14,926 ActorAddr-(T|:38069)/PID:5198 esrally.metrics INFO Creating file race store
> 2020-02-27 11:09:14,926 ActorAddr-(T|:38069)/PID:5198 esrally.mechanic.mechanic WARNING Cannot store system metrics: ('No race with race id [b50d6f7e-a699-4d8d-a894-4dd27713b510]', None).
> 2020-02-27 11:09:14,926 ActorAddr-(T|:38069)/PID:5198 esrally.metrics INFO Closing metrics store.
> 2020-02-27 11:09:14,928 ActorAddr-(T|:33407)/PID:5196 esrally.actor ERROR Child actor exited with [ChildActorExited:ActorAddr-(T|:41233)] while in status [starting].
> 2020-02-27 11:09:14,943 ActorAddr-(T|:34751)/PID:5195 esrally.actor INFO BenchmarkActor received unknown message [ChildActorExited:ActorAddr-(T|:33407)] (ignoring).
> 2020-02-27 11:09:17,926 -not-actor-/PID:5186 esrally.rally INFO Attempting to shutdown internal actor system.
> 2020-02-27 11:09:17,928 -not-actor-/PID:5194 root INFO ActorSystem Logging Shutdown
> 2020-02-27 11:09:17,949 -not-actor-/PID:5193 root INFO ---- Actor System shutdown
> 2020-02-27 11:09:17,950 -not-actor-/PID:5186 esrally.rally INFO Actor system is still running. Waiting...
> 2020-02-27 11:09:18,952 -not-actor-/PID:5186 esrally.rally INFO Shutdown completed.
> 2020-02-27 11:09:18,953 -not-actor-/PID:5186 esrally.rally ERROR Cannot run subcommand [race].
> Traceback (most recent call last):
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/rally.py", line 714, in dispatch_sub_command
> race(cfg)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/rally.py", line 624, in race
> with_actor_system(racecontrol.run, cfg)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/rally.py", line 652, in with_actor_system
> runnable(cfg)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 365, in run
> raise e
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 362, in run
> pipeline(cfg)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 79, in __call__
> self.target(cfg)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 290, in from_distribution
> return race(cfg, distribution=True)
> File "/home/spayne/.local/lib/python3.6/site-packages/esrally/racecontrol.py", line 250, in race
> raise exceptions.RallyError(result.message, result.cause)
> esrally.exceptions.RallyError: (Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct., 'Traceback (most recent call last):\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 360, in fetch\n net.download(download_url, distribution_path, progress_indicator=progress)\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 152, in download\n expected_size_in_bytes = download_http(url, tmp_data_set_path, expected_size_in_bytes, progress_indicator)\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/utils/net.py", line 116, in download_http\n raise urllib.error.HTTPError(url, r.status, "", None, None)\nurllib.error.HTTPError: HTTP Error 404: \n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 565, in receiveMsg_StartNodes\n self.mechanic.start_engine()\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/mechanic.py", line 676, in start_engine\n binaries = self.supply()\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 218, in __call__ \n supplier.fetch()\n File "/home/spayne/.local/lib/python3.6/site-packages/esrally/mechanic/supplier.py", line 366, in fetch\n "version [%s] is correct." % (download_url, self.version))\nesrally.exceptions.SystemSetupError: (\'Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.6.0-linux-x86_64.tar.gz]. Please check that the specified version [6.6.0] is correct.\', None)\n')
> 
> ```

---

<div class="post-metadata">

**Author:** ![sipayne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sipayne/32/63416_2.png) [@sipayne](https://discuss.elastic.co/u/sipayne)\
**Post date:** [February 27, 2020, 12:06pm UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207/2 "2020-02-27T12:06:37Z")

</div>

Ahah! I found that the URL template was configured in my `$TEAM_PATH/v1/vanilla/config.ini` where `$TEAM_PATH =~/elasticsearch-src/teams`. This line in particular:

```auto
jdk.bundled.release_url = https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-{{VERSION}}-{{OSNAME}}-{{ARCH}}.tar.gz

```

when I changed it to remove the `-{{OSNAME}}-{{ARCH}}` the download worked.

Phew! On to the next challenge...

---

<div class="post-metadata">

**Author:** ![baz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/baz/32/5183_2.png) [@baz](https://discuss.elastic.co/u/baz)\
**Post date:** [March 2, 2020, 9:30pm UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207/3 "2020-03-02T21:30:37Z")

</div>

Great sleuthing @sipayne! When in doubt, test out a rally run without any external additions to see if it also fails. I typically grab the generic "run a race" command from our docs. Feel free to ask us if you have any other questions and welcome to the community.

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [March 3, 2020, 12:08pm UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207/4 "2020-03-03T12:08:24Z")

</div>

Hi,

just as an additional info: Is it possible that you cloned [https://github.com/elastic/rally-teams](https://github.com/elastic/rally-teams) and based your custom team configuration on the `master` branch of that repo? We have a [versioning scheme](https://github.com/elastic/rally-teams#versioning-scheme) to support a wide range of Elasticsearch versions and if you look in branch [`6`](https://github.com/elastic/rally-teams/blob/6/cars/v1/vanilla/config.ini), you'll notice that the download URL is correct for version 6.x:

```auto
# deprecated
release_url = https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-{{VERSION}}.tar.gz
# new
jdk.unbundled.release_url = https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-{{VERSION}}.tar.gz

```

Daniel

---

<div class="post-metadata">

**Author:** ![sipayne](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sipayne/32/63416_2.png) [@sipayne](https://discuss.elastic.co/u/sipayne)\
**Post date:** [March 3, 2020, 3:47pm UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207/5 "2020-03-03T15:47:52Z")

</div>

Hi @danielmitterdorfer,

Actually I didn't know about rally-teams, thank you for letting me know, it might have saved me some struggles. I copied `.rally/benchmarks/teams/default` but since that is a git repo it has its own versioning, and I may have started from the master branch rather than branch 6. I'm a bit hazy in my recollection what I did!

Simon

---

<div class="post-metadata">

**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\
**Post date:** [March 4, 2020, 1:38pm UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207/6 "2020-03-04T13:38:45Z")

</div>

Hi Simon,

> [@sipayne](#):
>
> . I copied `.rally/benchmarks/teams/default` but since that is a git repo

This is actually your local checkout of [GitHub - elastic/rally-teams: Default Elasticsearch configurations for the Elasticsearch benchmarking tool Rally](https://github.com/elastic/rally-teams). Our docs about so-called [team repositories](https://esrally.readthedocs.io/en/stable/car.html#custom-team-repositories) explain their usage in a bit more detail.

Daniel

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 1, 2020, 1:38pm UTC](https://discuss.elastic.co/t/cannot-download-es-distribution-6-6-0-as-rally-builds-incorrect-download-path/221207/7 "2020-04-01T13:38:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.