Racing the patched 6.4.0 source

Mikhail_Khludnev · January 24, 2019, 11:52am

Hello,

Thanks for the great tool. I'm trying to benchmark patched 6.4.0 source, thus I use --revision=current, but it seems like it uses master branch for teams config (~/.rally/benchmarks/teams/default) that pull 7.0 configs which breaks 6.4.0 ES. How I can specify to use older teams config.
Your suggestions are appreciated.

danielmitterdorfer · January 24, 2019, 12:15pm

Hi,

glad to hear you like Rally. Yes, you are right: the logic in Rally assumes that benchmarks that build Elasticsearch from the source distribution are against master (or a commit that is very close to that). You can override this logic by explicitly specifying --distribution-version=6.4.0 as an additional command line parameter. Then Rally should checkout the correct branches for in the teams and the track repository.

Daniel

Mikhail_Khludnev · January 24, 2019, 2:05pm

I did that, but after that it withdraw my jdk11 which I had in JAVA_HOME and asked for earlier ones, which make me think that it ignores --revision and doesn't build. I'll doublecheck .
Thanks for your response.

danielmitterdorfer · January 25, 2019, 1:31pm

Hi,

I agree that the build requirements for Elasticsearch can be a little bit hard to understand at first especially since the release cadence of Java itself has increased. To build Elasticsearch 6.4 you need Java 10 (see e.g. the contribution guidelines on the 6.4 branch). Rally respects those requirements in the rally-teams configuration so we use the correct JDK versions depending on the Elasticsearch version that you want to benchmark. I hope that makes sense.

Daniel

Mikhail_Khludnev · January 27, 2019, 7:34pm

$ esrally --distribution-version=6.4.0 --track-path=track.json  --revision=current  

 INFO Checking out [6] in .. /benchmarks/teams/default] for distribution version [6.4.0].
 INFO Rebasing on [6] in [...

I like it. It works.

INFO Resolved download URL [https://artifacts...
INFO Skipping download for version [6.4.0]. Found an existing binary at [

It refuses to build and picks the distro. It's an issue.
I don't think it's really demanding, and I'll switch to master for my experiments.

danielmitterdorfer · January 28, 2019, 7:26am

Hi,

I was not completely precise before. As you provide both a revision and a distribution version Rally implicitly picks the distribution in this case but you can explicitly override that choice. The following will do what you want:

esrally --pipeline=from-sources-complete --distribution-version=6.4.0 --track-path=track.json  --revision=current

Daniel

Mikhail_Khludnev · February 23, 2019, 1:22pm

    $ esrally  --track-path=track.json  --revision=current  --preserve-install true --car="4gheap"  --client-options="timeout:6000" --pipeline=from-sources-complete --distribution-version=6.4.0
[ERROR] Cannot race. --distribution-version can only be used together with pipeline from-distribution, but you specified from-sources-complete.

I'd rather move to development in trunk.

system · March 23, 2019, 1:22pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.