Elasticsearch 7.17.7 does not start - "java.security.AccessControlException: access denied"

[jacotec]

Hi,

I'm despairing with Elasticsearch on one of my Ubuntu 20.04 LTS VM's. I've just installed ES 7.17.7 from the official repo, did no config changes or something else, just did

apt install elasticsearch
systemctl daemon-reload
systemctl enable elasticsearch
sytemctl start elasticsearch

ES 7.17.7 does not start!

In the ES logs it's always showing this error message which does not give me any results at Google:

...
[2022-11-02T15:19:49,275][INFO ][o.e.n.Node               ] [majak] stopping ...
[2022-11-02T15:19:49,288][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [majak] [controller/1465] [Main.cc@174] ML controller exiting
[2022-11-02T15:19:49,289][INFO ][o.e.x.m.p.NativeController] [majak] Native controller process has stopped - no new native processes can be started
[2022-11-02T15:19:49,290][INFO ][o.e.x.w.WatcherService   ] [majak] stopping watch service, reason [shutdown initiated]
[2022-11-02T15:19:49,291][INFO ][o.e.x.w.WatcherLifeCycleService] [majak] watcher has stopped and shutdown
[2022-11-02T15:19:49,307][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [majak] uncaught exception in thread [process reaper (pid 1465)]
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "modifyThread")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:485) ~[?:?]
	at java.security.AccessController.checkPermission(AccessController.java:1068) ~[?:?]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:411) ~[?:?]
	at org.elasticsearch.secure_sm.SecureSM.checkThreadAccess(SecureSM.java:160) ~[?:7.17.7]
	at org.elasticsearch.secure_sm.SecureSM.checkAccess(SecureSM.java:120) ~[?:7.17.7]
	at java.lang.Thread.checkAccess(Thread.java:2360) ~[?:?]
	at java.lang.Thread.setDaemon(Thread.java:2308) ~[?:?]
	at java.lang.ProcessHandleImpl.lambda$static$0(ProcessHandleImpl.java:103) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:637) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:928) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1021) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) [?:?]
	at jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186) ~[?:?]
[2022-11-02T15:19:49,749][INFO ][o.e.n.Node               ] [majak] stopped
[2022-11-02T15:19:49,749][INFO ][o.e.n.Node               ] [majak] closing ...
[2022-11-02T15:19:49,806][INFO ][o.e.h.AbstractHttpServerTransport] [majak] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}

I highly appreciate any ideas what's going on here.

Thanks!

[warkolm (Mark Walkom)]

Welcome to our community!

Can you post all of your Elasticsearch log and your elasticsearch.yml please.

[Unitiser]

As stated by jacotec, the issue is present with the version 7.17.7 of elasticsearch on Ubuntu 20.04 LTS with the default configurations. I had the same issue using the LXD container for Ubuntu 20.04.

Reading the patchnotes, I noticed that the bundled JDK has been bumped from 18 to 19 in this release. Since we are dealing with an obscure java exception, my intuition tells me that this is the root of the issue.

To test this theory, I used /etc/default/elasticsearch to define ES_JAVA_HOME=/usr/lib/jvm/default-java (which is provided by apt install default-jre ). As i suspected, the error goes away and we can start the service.

Hopefully this will help the ES team to troubleshoot the issue and, in the meantime, provide the community with a quick workaround.

[theglamolondon (GlamoLondon)]

I confirm that. I updated java version from 8 to 11 and it's work for me.
thanks @Unitiser

[jacotec]

Ahm, just to light my confusion: ES should use the bundled Java which comes with it, not the system Java?!? So how does an update of the system Java help?

Or am I completely on the wrong track here?

[Unitiser]

You are right, in the best of world, we would use the SDK bundled with ES. However, last patch, they upgraded the SDK bundled with the app and it causes some issues on our particular OS version. What i did here is to downgrade the JDK to the one that was supported when Ubuntu freezed the LTS release. In my case, default-jre is an alias for the package openjdk-11-jre.

To answer your question, the workaround is not to upgrade the system JDK. It's to revert the JDK upgrade that caused the issue in the first place. To do that, you can specify the environment variable ES_JAVA_HOME to tell the app which SDK to use. Luckily, the file /etc/default/elasticsearch can be used to define environment variable for ES. In my case, I chose to use the JDK that was already on my system : /usr/lib/jvm/default-java.

That being said, this is a workaround. I expect the ES team to figure out the source of the issue and patch the code accordingly at some point. So if you don't want to deal with this at all, you can simply use ES 7.17.6 instead. In my case, ES would throw an error saying it was unable to downgrade from 7.17.7 to 7.17.6, so I didn't have much choice.