# Esrally configure does not find jdk on AWS linux (centos based) **URL:** \ **Category:** Elasticsearch\ **Tags:** rally\ **Created:** [November 3, 2016, 8:38pm UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946 "2016-11-03T20:38:44Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Alexander\_Gray\_II](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexander_gray_ii/32/1669_2.png) [@Alexander\_Gray\_II](https://discuss.elastic.co/u/Alexander_Gray_II)\ **Post date:** [November 3, 2016, 8:38pm UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/1 "2016-11-03T20:38:44Z")
When I run "esrally configure" it does not detect JDK8, which i have installed. # yum list installed | grep jdk java-1.7.0-openjdk.x86\_64 1:1.7.0.111-2.6.7.2.68.amzn1 installed java-1.8.0-openjdk.x86\_64 1:1.8.0.111-1.b15.25.amzn1 @amzn-updates java-1.8.0-openjdk-devel.x86\_64 1:1.8.0.111-1.b15.25.amzn1 @amzn-updates java-1.8.0-openjdk-headless.x86\_64 1:1.8.0.111-1.b15.25.amzn1 @amzn-updates # java -version openjdk version "1.8.0\_111" OpenJDK Runtime Environment (build 1.8.0\_111-b15) OpenJDK 64-Bit Server VM (build 25.111-b15, mixed mode) Looking at this file, it looks for /etc/alternatives/java\_sdk\_1.%d.0 > But on this os (AWS linux) /etc/alternatives/java\_sdk\_1.8.0 points to /usr/lib/jvm/java-1.8.0-openjdk.x86\_64 ls -ltrh /etc/alternatives/java\_sdk\_1.8.0 lrwxrwxrwx 1 root root 38 Nov 3 20:24 /etc/alternatives/java\_sdk\_1.8.0 -\> /usr/lib/jvm/java-1.8.0-openjdk.x86\_64 and then /usr/lib/jvm/java-1.8.0-openjdk.x86\_64 to java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86\_64 ls -ltrh /usr/lib/jvm/java-1.8.0-openjdk.x86\_64 lrwxrwxrwx 1 root root 50 Nov 3 19:48 /usr/lib/jvm/java-1.8.0-openjdk.x86\_64 -\> java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86\_64 And that double pointing seem trip it up (i think). Nonetheless, I enter "/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86\_64" as the java home but when I run "esrally --distribution-version=5.0.0" I get a huge error: ERROR: Cannot race Reason: This race ended early with a fatal crash. For details please see the logs. And in the logs i see: 2016-11-03 20:34:27,464 root ERROR Cannot run subcommand [race]. Traceback (most recent call last): File "/usr/local/lib/python3.4/site-packages/esrally/config.py", line 109, in opts return self.\_opts[self.\_k(scope, section, key)] KeyError: 'source::local.src.dir' Any ideas how to get this to work? ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [November 4, 2016, 6:43am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/2 "2016-11-04T06:43:40Z")
Hi @Alexander_Gray_II, > [@Alexander\_Gray\_II](#): > > that double pointing seem trip it up (i think). Yes, this is definitely the cause of the problem. Could you please [open a ticket on Github](https://github.com/elastic/rally/issues)? > [@Alexander\_Gray\_II](#): > > Any ideas how to get this to work? Yes, you are missing a configuration value but it seems to me we can reconstruct it together. Can you please run: ```auto cat ~/.rally/rally.ini ``` and paste the output? Please also check that you scrub the values for `datastore.user` and `datastore.password` before posting in case they are set. Daniel ---
**Author:** ![Alexander\_Gray\_II](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexander_gray_ii/32/1669_2.png) [@Alexander\_Gray\_II](https://discuss.elastic.co/u/Alexander_Gray_II)\ **Post date:** [November 4, 2016, 11:17am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/3 "2016-11-04T11:17:16Z")
Here is my rally.ini: ```auto # cat /root/.rally/rally.ini [meta] config.version = 6 [system] root.dir = /root/.rally/benchmarks log.root.dir = logs env.name = local [provisioning] local.install.dir = install [runtime] java8.home = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86_64 [benchmarks] local.dataset.cache = ${system:root.dir}/data [reporting] datastore.type = in-memory datastore.host = datastore.port = datastore.secure = datastore.user = datastore.password = [tracks] default.url = https://github.com/elastic/rally-tracks [defaults] preserve_benchmark_candidate = False ``` Here is the output of the **esrally configure** command: ```auto # esrally configure ________ / __\____ _/ / /_ __ / /_/ / __ `/ / / / / / / _, _/ /_/ / / / /_/ / /_/ |_|\ __,_/_/_/\__ , / / ____ / Running simple configuration. You can run the advanced configuration with: esrally configure --advanced-config [✓] Autodetecting available third-party software which: no gradle in (/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin) git : [✓] gradle : [✕] JDK 8 : [✕] (You cannot benchmark Elasticsearch 5.x without a JDK 8 installation) ********************************************************************************** You don't have the necessary software to benchmark source builds of Elasticsearch. You can still benchmark binary distributions with e.g.: esrally --pipeline=from-distribution --distribution-version=5.0.0-alpha5 See https://esrally.readthedocs.io/en/latest/pipelines.html#from-distribution ********************************************************************************** [✓] Setting up benchmark data directory in [/root/.rally/benchmarks] (needs several GB). Enter the JDK 8 root directory:: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.111-1.b15.25.amzn1.x86_64 [✓] Configuration successfully written to [/root/.rally/rally.ini]. Happy benchmarking! To benchmark Elasticsearch 5.0.0-alpha2 with the default benchmark run: esrally --pipeline=from-distribution --distribution-version=5.0.0-alpha2 For help, type esrally --help or see the user documentation at https://esrally.readthedocs.io ``` So the command that I was running that got the error is: ```auto ``` But the command that I SHOULD be running (by looking at the configure output) is: ```auto ``` And when I run THAT command it gets much much further. I get this error: ```auto ``` Ah, so I can't be root, so I switch back to ec2-user and re-run the command (I run **esrally configure** for ec2-user and the rally.ini file looks the same as roots) Now I get this error when running: ```auto ``` ```auto 2016-11-04 11:12:04,711 rally.launcher INFO ES launch: ['bin/elasticsearch', '-Enode.name=rally-node0', '-Epath.logs=/home/ec2-user/.rally/benchmarks/races/2016-11-04-11-11-51/local/logs/geonames/append-no-conflicts/server'] 2016-11-04 11:12:04,961 rally.launcher INFO rally-node0: starts elasticsearch 2016-11-04 11:12:04,961 rally.launcher INFO rally-node0: 2016-11-04 11:12:04,999 rally.launcher INFO rally-node0: Option Description 2016-11-04 11:12:04,999 rally.launcher INFO rally-node0: ------ ----------- 2016-11-04 11:12:04,999 rally.launcher INFO rally-node0: -E Configure an Elasticsearch setting 2016-11-04 11:12:04,999 rally.launcher INFO rally-node0: -V, --version Prints elasticsearch version 2016-11-04 11:12:04,999 rally.launcher INFO rally-node0: information and exits 2016-11-04 11:12:05,0 rally.launcher INFO rally-node0: -d, --daemonize Starts Elasticsearch in the background 2016-11-04 11:12:05,0 rally.launcher INFO rally-node0: -h, --help show help 2016-11-04 11:12:05,0 rally.launcher INFO rally-node0: -p, --pidfile Creates a pid file in the specified 2016-11-04 11:12:05,0 rally.launcher INFO rally-node0: path on start 2016-11-04 11:12:05,0 rally.launcher INFO rally-node0: -s, --silent show minimal output 2016-11-04 11:12:05,0 rally.launcher INFO rally-node0: -v, --verbose show verbose output 2016-11-04 11:12:05,0 rally.launcher INFO rally-node0: ERROR: Elasticsearch settings must be prefixed with [es.] but was [node.name] 2016-11-04 11:12:24,714 rally.launcher ERROR Could not start node 'rally-node0' within timeout period of 20.0 seconds. ``` I feel I'm so close. Any help will be appreciated. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [November 4, 2016, 11:57am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/4 "2016-11-04T11:57:10Z")
Hi @Alexander_Gray_II, Based on the output you've pasted here I guess you are on Rally 0.3.3 (current is 0.4.3); you can check this with `esrally --version`. Can you please upgrade it by running `pip3 install --upgrade esrally`? You don't have to reconfigure Rally. Just upgrade the package and with pip and you should be fine. Support for the Elasticsearch alpha releases has been removed in more recent versions of Rally, so you should now run: ```auto esrally --pipeline=from-distribution --distribution-version=5.0.0 ``` Daniel ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [November 4, 2016, 12:05pm UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/5 "2016-11-04T12:05:59Z")
I've now created: - [https://github.com/elastic/rally/issues/177](https://github.com/elastic/rally/issues/177) - [https://github.com/elastic/rally/issues/178](https://github.com/elastic/rally/issues/178) due to your feedback. The next release (0.4.4) is right around the corner so I scheduled them for 0.4.5. Daniel ---
**Author:** ![Alexander\_Gray\_II](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexander_gray_ii/32/1669_2.png) [@Alexander\_Gray\_II](https://discuss.elastic.co/u/Alexander_Gray_II)\ **Post date:** [November 4, 2016, 12:58pm UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/6 "2016-11-04T12:58:07Z")
Oh wow, thanks Daniel! You're really on top of this! I verified that I do indeed have esrally: ```auto $ esrally --version esrally 0.4.3 ``` I'm installing it via ansible too, and I tear down the ec2 instance after each run. This did the trick: ```auto esrally --pipeline=from-distribution --distribution-version=5.0.0 ``` Thank you so much (I still got an error OOM, but that is my own fault, since I'm running this on a tiny t2.small instance, which doesn't have enough memory) Sincerely, @Alexander_Gray_II ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [November 4, 2016, 1:04pm UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/7 "2016-11-04T13:04:01Z")
Hi @Alexander_Gray_II, I'm glad to hear that it works now! I also fixed both issues that I've mentioned above in the meantime so they'll be included in Rally 0.4.4. that I'll release next week. Daniel ---
**Author:** ![khou](https://avatars.discourse-cdn.com/v4/letter/k/848f3c/32.png) [@khou](https://discuss.elastic.co/u/khou)\ **Post date:** [November 9, 2016, 2:35am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/8 "2016-11-09T02:35:18Z")
Hi @danielmitterdorfer, I'm getting the same exact issue, with the latest esrally version (0.4.4) running on an AWS instance. I have also arrived at the OOM conclusion (t2.micro causes), but this is with 0.4.4 and the issue fix for following multiple symlinks. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [November 9, 2016, 7:39am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/9 "2016-11-09T07:39:31Z")
Hi @khou, 0.4.4. indeed fixed the multi-symlink issue so I wonder about the cause. Let me grab an AWS Linux EC2 instance later and figure out what's going on. Daniel ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [November 9, 2016, 9:02am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/10 "2016-11-09T09:02:07Z")
Hi @khou, I could reproduce the problem now. I don't know (yet) why it cannot determine the JDK directory but I will eventually release a fix. However, note that this is a _convenience_ feature and I do not expect to autodetect the JDK directory on every platform. It works perfectly fine if you enter the path to the JDK 8 directory. Daniel ---
**Author:** ![khou](https://avatars.discourse-cdn.com/v4/letter/k/848f3c/32.png) [@khou](https://discuss.elastic.co/u/khou)\ **Post date:** [November 9, 2016, 9:15am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/11 "2016-11-09T09:15:51Z")
@danielmitterdorfer Understand! Thank you very much for the swift replies! I will manually enter the path for now :)! ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [November 9, 2016, 9:32am UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/12 "2016-11-09T09:32:19Z")
Hi @khou, great! Thanks for your report though. It's fixed on master now and will be released with Rally 0.4.5. The problem was that the JDK detection code thought it was running on Debian. Daniel ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:06pm UTC](https://discuss.elastic.co/t/esrally-configure-does-not-find-jdk-on-aws-linux-centos-based/64946/13 "2017-07-05T22:06:03Z")