Esrally will hang if I use elasticsearch 8.0.0-alpha1

Elastic Stack Elasticsearch
1

Hi,
My OS is Ubuntu 21.04. esrally will hang when I tried the following command.

esrally --pipeline=from-distribution --distribution-version=8.0.0-alpha1 --track=geonames --challenge=append-no-conflicts --include-tasks="delete-index"

The last several lines of the log "~/.rally/logs/rally.log":
2021-10-31 17:03:54,889 ActorAddr-(T|:37489)/PID:124618 esrally.actor INFO Cluster consisting of [{'host': '127.0.0.1', 'port': 39200}] will be provisioned by Rally.
2021-10-31 17:03:54,890 ActorAddr-(T|:37489)/PID:124618 esrally.actor INFO Capabilities [{'coordinator': True, 'ip': '127.0.0.1', 'Convention Address.IPv4': '127.0.0.1:1900', 'Thespian ActorSystem Name': 'multiprocTCPBase', 'Thespian ActorSystem Version': 2, 'Thespian Watch Supported': True, 'Python Version': (3, 9, 5, 'final', 0), 'Thespian Generation': (3, 9), 'Thespian Version': '1635699831468'}] match requirements [{}].
2021-10-31 17:03:54,905 ActorAddr-(T|:44235)/PID:124638 esrally.actor INFO Capabilities [{'coordinator': True, 'ip': '127.0.0.1', 'Convention Address.IPv4': '127.0.0.1:1900', 'Thespian ActorSystem Name': 'multiprocTCPBase', 'Thespian ActorSystem Version': 2, 'Thespian Watch Supported': True, 'Python Version': (3, 9, 5, 'final', 0), 'Thespian Generation': (3, 9), 'Thespian Version': '1635699831468'}] match requirements [{'coordinator': True}].

The console output is:
[INFO] Preparing for race ...

I tried using 7.0.0 and it can complete successfully. Moreover, if I tried using version "7.15.1", it will report 'Cannot download Elasticsearch distribution from [https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss
-7.15.1-linux-x86_64.tar.gz]. Please check that the specified version [7.15.1] is correct.' Where can I get the latest support elasticsearch version?

2

There is no OSS distribution anymore, the last one was 7.10.2, for any version above that you will need to use the elastic distribution with the elastic license.

3

Thanks a lot for the information. Then can I use the latest Elasticsearch(8.0.0 or above)? I tried using the 8.0.0-alpha, the Elasticsearch could be downloaded successfully, but the esrally will always hang. Does anyone know the reason of the issue? How can I solve it?

4
  1. The alpha2 has been out for a while, so I would try that one (though things around security have changed and it will enable security by default now)
  2. You might be a little early if you are using the releases (and not dev mode). If it's working great, but I don't think this is actively built for it yet. Somewhat related see update elasticsearch dependency by Ro0tk1t · Pull Request #1364 · elastic/rally · GitHub for some upgrading discussions even within 7.x.
5

Thanks a lot. So the current Elasticsearch GitHub code can neither be used, right?

6

In development mode it should work against master (see GitHub - elastic/rally: Macrobenchmarking framework for Elasticsearch), but it's quite fast moving, so what might work with the state of alpha1 or 2 of Elasticsearch I'm not sure of.

7

Hi @shankang, what version of Rally are you using? This can be retrieved with esrally --version

8

My esrally version is 1.4.0.

9

Because of changes to the distribution naming schemes we had to update some code somewhere in the Rally 2.x era. I highly recommend you update to our latest version, esrally==2.3.0

10

Thank you very much. The latest esrally seems to work normally now.
I'm trying to optimize the Elasticsearch. Is there anyway to make esrally to use my developping Elasticsearch?

11

You mean you're making changes to Elasticsearch and want to benchmark them?

Can you please open a new topic if you have further questions?

12

Thank you very much for your quick reponse. It really saves my lots of time. :slight_smile: