# Elasticsearch 7.16.2 failed to start when switching from Java 8 to Java 11 **URL:** \ **Category:** Elasticsearch\ **Created:** [April 3, 2022, 1:02am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404 "2022-04-03T01:02:22Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [April 3, 2022, 1:02am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/1 "2022-04-03T01:02:22Z")
I'm testing Elasticsearch (ES) single node cluster with Java 8 and Java 11 Java 8 and Java 11 are installed on Linux server RHEL release 7.9 in directories: /usr/java/jdk1.8.0\_281-amd64 includes subdirectories bin, include, jre, legal, lib, man /usr/java/jdk-11.0.14 includes subdirectories bin, conf, include, jmods, legal, lib, man Running java -version in cmd returns java version "1.8.0\_281" Java (TM) SE Runtime Environment (build 1.8.0\_\_281-b09) Java HotSpot (TM) 64-bit Server VM (build 25.281-b09, mixed mode) ES single node cluster has been setup and configured with x-pack security from archive package Elasticsearch-7.16.2-linux-x86\_64.tar.gz I'm starting es node with Java 8 (and without using bundled jdk) from launching script as follows ```auto ....................................................... export PATH=/usr/java/jdk1.8.0_281-amd64/bin export ES_KEYSTORE_PASSPHRASE_FILE=/path_to_file/es-keystore-sec.txt cat /path_to_file/es-keystore-sec.txt | ES_JAVA_OPTS=-Des.search.ignore_awareness_attributes=true -Xms4g -Xmx4g bin/elasticsearch -d -p /path_to_pid_file/es-pid > /dev/null 2>&1 & ``` Node starts gracefully with the appropriate entries in log file when referencing Java 8 in the script When I switch from Java 8 to Java 11 and changed the 'jdk1.8.0\_281-amd64' for 'jdk-11.0.14' in the above script and run it then the node doesn't start and nothing even written in log file Am I missing something in Java 11 configuration or ES\_JAVA\_OPTS, or something else need to be changed in the above launching script to conform Java 11? I noted Java 11 doesn't include JRE. Should I download and install it too? Any help, suggestion on this issue resolution will be appreciated Thanks in advance ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [April 15, 2022, 8:22pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/2 "2022-04-15T20:22:23Z")
The problem and the solution was in jvm.options default settings for java 9 and higher garbage collection logging `# JDK 9+ GC logging 9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m` When I switched the cluster from java 8 to java 11 than the jvm and cluster didn't start with above default setting and the log file even hasn't been created. The solution was either to specify the full path to gc.log `9-:-Xlog:gc*,gc+age=trace,safepoint:file=/full_path/logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m` or export environment variable ES\_TMPDIR from launching script and specify it in jvm.options `9-:-Xlog:gc*,gc+age=trace,safepoint:file=${ES_TMPDIR}/logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m` Hope this might help someone with similar problem ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [April 17, 2022, 4:50pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/3 "2022-04-17T16:50:58Z")
I think it might be reasonable to place a note in elastic docs like "... if you're using java 9+ (or switch your cluster from java 8 to java 9+) and you specify the path to log (not default) in Elasticsearch.yml than you must change in the jvm.options line '9-:-Xlog....' and specify a path to the gc.log with either your full path or with ES\_TMPDIR. Othervise after launchig the cluster, the jvm will not start, elastic instance will not start and the log file will not be created at all." It took me some time to figure this behavior out and might be helpfull for elastic users. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 17, 2022, 5:35pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/4 "2022-04-17T17:35:11Z")
> [@dzyubanv](#): > > I think it might be reasonable to place a note in elastic docs Mmmaybe. I mean it's technically a supported configuration, but it's definitely not a recommended one. For instance [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/advanced-configuration.html) say: > Modifying advanced settings (i.e. JVM options) is generally not recommended and could negatively impact performance and stability. Using the Elasticsearch-provided defaults is recommended in most circumstances. and [these docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup.html#jvm-version) say: > The bundled JVM is the recommended JVM ... You're doing two not-recommended things by running (a) Java 11 with (b) non-default options. Your problem was really a JDK config issue for which you should consult the docs for your chosen (nonstandard) JDK. Advanced JDK config docs doesn't really seem like something in scope for the Elasticsearch manual. ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [April 18, 2022, 7:59pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/5 "2022-04-18T19:59:04Z")
Thanks for the response David. I got your notes about using bundled java and elastic default settings but still have a few questions about configuration. I had to use Java 11 to follow up the firm's requirement. To make the cluster with es-7.16.2 and java 11 functional I had to change the root /config/jvm.options as described above. I know from docs the changes need to be done in /config/jvm.options.d/jvm.options instead of changing root /config/jvm.options I tried this use case: left the root /config/jvm.options unchangeable, copied it into /config/jvm.options.d/jvm.options, made the change in it but in this case the node is not starting at all and not writing anything in the log. Whatever worked for me in the root /config/jvm.options, doesn't work in /config/jvm.options.d/jvm.options _**Am I missing something else in configuration ? Should be file name in directory /config/jvm.options.d/ somehow differ from file name in root directory /config/ ?**_ _ **Are there any other options how to specify the path to gc.logs instead of changing jvm.options ?** _ I raised these questions on the forum already at [Overriding default jvm options is not working (elastic 7.16.2 and java 11)](https://discuss.elastic.co/t/overriding-default-jvm-options-is-not-working-elastic-7-16-2-and-java-11/302525) with no answer yet. Appreciate your help and attention Thanks in advance ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 19, 2022, 7:19am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/6 "2022-04-19T07:19:14Z")
I'd expect problems starting up the JVM to be reported to `stderr` (perhaps `stdout`) rather than being written to the log file. We can't write to any log files before starting the JVM. Elasticsearch should read JVM options from any file whose name ends in `.options` (i.e. it searches for `$ES_CONF/jvm.options.d/*.options`). The GC logs are written by the JVM itself so any changes need to be set as a JVM option. ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [April 20, 2022, 8:54pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/7 "2022-04-20T20:54:44Z")
Thanks for the response David. I tested overriding default JVM options on single node cluster and Java 11. Root jvm options file jvm.options resides in directory /config/ For custom jvm options I tested 2 files - jvm.options and java11-gc-logging.options reside in directory /config/jvm.options.d/ The node started gracefully and the cluster health was green when in root jvm options file /config/jvm.options the default line `9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m` has been commented out and either in /config/jvm.options.d/jvm.options or in /config/jvm.options.d/java11-gc-logging.options I added the following line `9-:-Xlog:gc*,gc+age=trace,safepoint:file=${ES_TMPDIR}/logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m` Any others use cases didn't work for me. Is this normal behavior of overriding default jvm options or I'm still missing something ? Thanks in advance ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 21, 2022, 6:47am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/8 "2022-04-21T06:47:00Z")
I would expect the default to work too, but maybe there's something else unusual in your environment that prevents Elasticsearch writing to `logs/gc.log`? When it didn't work, what did the process report to `stderr` and `stdout`? ---
**Author:** ![dzyubanv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dzyubanv/32/24110_2.png) [@dzyubanv](https://discuss.elastic.co/u/dzyubanv)\ **Post date:** [April 21, 2022, 9:30pm UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/9 "2022-04-21T21:30:09Z")
Hi David, Here is what I did Didn't change the root jvm options /config/jvm.options and it contains the original line `9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m` Created custom jvm options /config/jvm.options.d/gc.options being contained lines ```auto 9-:-Xlog:disable 9-:-Xlog:all=warning:stderr:utctime,level,tags 9-:-Xlog:gc*,gc+age=trace,safepoint:file=${ES_TMPDIR}/gc.log:utctime,pid,tags:filecount=32,filesize=64m ``` The launching script to start the node constructs and exports ${ES\_TMPDIR} dynamically pointing to the starting node's logs The node doesn't start and the log is not even created. Can you please let me know what I'm doing wrong to being able to write to stdout or/and stderror and which location should I look at ? Many thanks ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [April 22, 2022, 8:16am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/10 "2022-04-22T08:16:03Z")
Sorry, I don't know. I've no idea what could prevent Elasticsearch writing to the console. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 20, 2022, 8:16am UTC](https://discuss.elastic.co/t/elasticsearch-7-16-2-failed-to-start-when-switching-from-java-8-to-java-11/301404/11 "2022-05-20T08:16:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.