Problem running my first benchmark with esrally - rally.launcher cannot start node


#1

Hi.
I am new to ES and Rally. I'm trying to educate myself on benchmarking ES with Rally. I'm following documentation on running a benchmark with rally.

Here is how I run the esrally command.

[elasticsearch@fedoravm ~]$ esrally --pipeline=from-distribution --distribution-version=5.0.0-alpha1

Here is the output on the command line:

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
            /____/


Writing additional logs to /home/elasticsearch/.rally/benchmarks/races/2016-09-30-15-35-01/local/logs/rally_out.log




Downloading Elasticsearch 5.0.0-alpha1 ...
Cloning into '/home/elasticsearch/.rally/benchmarks/tracks/default'...
remote: Counting objects: 370, done.
remote: Total 370 (delta 0), reused 0 (delta 0), pack-reused 370
Receiving objects: 100% (370/370), 59.41 KiB | 0 bytes/s, done.
Resolving deltas: 100% (262/262), done.
Checking connectivity... done.
Downloading data from http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geonames/documents.json.bz2 (189 MB) ... Done
Decompressing /home/elasticsearch/.rally/benchmarks/data/geonames/documents.json.bz2 (resulting size: 2.60 GB) ... Done
Racing on track [geonames] and challenge [append-no-conflicts] with car [defaults]
Rally will wipe the benchmark candidate directory [/home/elasticsearch/.rally/benchmarks/races/2016-09-30-15-35-01/local/tracks/geonames/append-no-conflicts/install] after the benchmark.


ERROR: Cannot race

Reason: This race ended early with a fatal crash. For details please see the logs.

Please check the log file [/home/elasticsearch/.rally/benchmarks/races/2016-09-30-15-35-01/local/logs/rally_out.log] for further details first.

If you need further help, please check Rally's docs at https://esrally.readthedocs.io or ask a question in the forum at https://discuss.elastic.co/c/rally.

If you think this is a bug, please file a report at https://github.com/elastic/rally/issues and include the log file for this race (/home/elasticsearch/.rally/benchmarks/races/2016-09-30-15-35-01/local/logs/rally_out.log).

I've checked the rally_out.log, and here are all messages with keyword error.

[elasticsearch@fedoravm ~]$ grep -i error /data/rally/benchmarks/races/2016-09-30-15-35-01/local/logs/rally_out.log 
2016-09-30 15:41:09,235 rally.launcher INFO rally-node0: ERROR: Elasticsearch settings must be prefixed with [es.] but was [node.name]
2016-09-30 15:41:28,798 rally.launcher ERROR Could not start node 'rally-node0' within timeout period of 20.0 seconds.
2016-09-30 15:41:28,798 root ERROR Cannot run subcommand [race].
raise exceptions.LaunchError("%s Please check the logs in '%s' for more details." % (msg, log_dir))
esrally.exceptions.LaunchError: Could not start node 'rally-node0' within timeout period of 20.0 seconds. Please check the logs in '/home/elasticsearch/.rally/benchmarks/races/2016-09-30-15-35-01/local/logs' for more details.
raise exceptions.RallyError("This race ended early with a fatal crash. For details please see the logs.").with_traceback(tb)
raise exceptions.LaunchError("%s Please check the logs in '%s' for more details." % (msg, log_dir))
esrally.exceptions.RallyError: This race ended early with a fatal crash. For details please see the logs.

Lastly, here is my rally.ini file:

elasticsearch@fedoravm ~]$ cat /data/rally/rally.ini 
[meta]
config.version = 6

[system]
root.dir = /home/elasticsearch/.rally/benchmarks
log.root.dir = logs
env.name = local

[source]
local.src.dir = /home/elasticsearch/.rally/benchmarks/src
remote.repo.url = https://github.com/elastic/elasticsearch.git

[build]
gradle.bin = /usr/bin/gradle

[provisioning]
local.install.dir = install

[runtime]
java8.home = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.77-1.b03.fc23.x86_64

[benchmarks]
local.dataset.cache = ${system:root.dir}/data

[reporting]
datastore.type = in-memory
datastore.host = 
datastore.port = 
datastore.secure = 
datastore.user = 
datastore.password = 

[tracks]
default.url = https://github.com/elastic/rally-tracks

[defaults]
preserve_benchmark_candidate = False

I'd appreciate any help to troubleshoot this.
Uncas


(Daniel Mitterdorfer) #2

Hi @uncas,

I've recently removed support for earlier Alpha versions of Elasticsearch. If you try with later alpha builds of Elasticsearch everything should work fine. E.g.:

esrally --pipeline=from-distribution --distribution-version=5.0.0-alpha5

Daniel


#3

Thank you Daniel. I will try that today, and will post results.


#4

Worked!

Thank you.


(Daniel Mitterdorfer) #5

Great! You're welcome.


(system) #6