Elasticsearch 8.5.1 does not start - “java.security.AccessControlException: access denied”

hrQAQ · November 28, 2022, 6:33pm

Hi,

I meet problem while starting elasticsearch. When I start elasticsearch it generates a java.security.AccessControlException: access denied

The error log and my elasticsearch.yml are list as follow.

Environment

OS: Centos 7.9

Elasticsearch version: 8.5.1

My Attempts

I tried to change network-host: 0.0.0.0 to network-host: 127.0.0.1, then it works well. But with this configuration, only the intranet can access elasticsearch. I have some external applications that need to access the es service within the server. which means, I wish to access elasticseach with：
```
http://<my_server_ip>:9200
```
I tried to set xpack.security.enabled: false, then it work well. But with this configuration, All people who know my ip can access my elasticsearch (which is very dangerous). I have set a password for elasticsearch, but this needs to enable xpack.security

I really don't know how to deal with it. Your help is greatly needed!

elasticsearch.yml

cluster.name: IShare
node.name: node-1
path.data: /data/openalex/elasticsearch/data
path.logs: /data/openalex/elasticsearch/logs
network.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["node-1"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: false
http.cors.enabled: true
http.cors.allow-origin: "*"

Total Error logs

[2022-11-29T01:37:06,259][INFO ][o.e.n.Node               ] [node-1] version[8.5.1], pid[2286], build[tar/c1310c45fc534583afe2c1c03046491efba2bba2/2022-11-09T21:02:20.169855900Z], OS[Linux/3.10.0-1160.80.1.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/19.0.1/19.0.1+10-21]
[2022-11-29T01:37:06,264][INFO ][o.e.n.Node               ] [node-1] JVM home [/usr/local/elasticsearch-8.5.1/jdk], using bundled JDK [true]
[2022-11-29T01:37:06,264][INFO ][o.e.n.Node               ] [node-1] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=ALL-UNNAMED, -XX:+UseG1GC, -Djava.io.tmpdir=/tmp/elasticsearch-8594388477649810696, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms31744m, -Xmx31744m, -XX:MaxDirectMemorySize=16642998272, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=25, -Des.distribution.type=tar, --module-path=/usr/local/elasticsearch-8.5.1/lib, --add-modules=jdk.net, -Djdk.module.main=org.elasticsearch.server]
[2022-11-29T01:37:07,642][INFO ][c.a.c.i.j.JacksonVersion ] [node-1] Package versions: jackson-annotations=2.13.2, jackson-core=2.13.2, jackson-databind=2.13.2.2, jackson-dataformat-xml=2.13.2, jackson-datatype-jsr310=2.13.2, azure-core=1.27.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot
[2022-11-29T01:37:08,692][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [aggs-matrix-stats]
[2022-11-29T01:37:08,693][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [analysis-common]
[2022-11-29T01:37:08,693][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [apm]
[2022-11-29T01:37:08,693][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [constant-keyword]
[2022-11-29T01:37:08,693][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [data-streams]
[2022-11-29T01:37:08,693][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [frozen-indices]
[2022-11-29T01:37:08,694][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [ingest-attachment]
[2022-11-29T01:37:08,694][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [ingest-common]
[2022-11-29T01:37:08,694][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [ingest-geoip]
[2022-11-29T01:37:08,694][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [ingest-user-agent]
[2022-11-29T01:37:08,694][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [kibana]
[2022-11-29T01:37:08,695][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [lang-expression]
[2022-11-29T01:37:08,695][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [lang-mustache]
[2022-11-29T01:37:08,695][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [lang-painless]
[2022-11-29T01:37:08,695][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [legacy-geo]
[2022-11-29T01:37:08,695][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [mapper-extras]
[2022-11-29T01:37:08,698][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [mapper-version]
[2022-11-29T01:37:08,698][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [old-lucene-versions]
[2022-11-29T01:37:08,699][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [parent-join]
[2022-11-29T01:37:08,699][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [percolator]
[2022-11-29T01:37:08,699][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [rank-eval]
[2022-11-29T01:37:08,699][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [reindex]
[2022-11-29T01:37:08,699][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [repositories-metering-api]
[2022-11-29T01:37:08,699][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [repository-azure]
[2022-11-29T01:37:08,700][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [repository-encrypted]
[2022-11-29T01:37:08,700][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [repository-gcs]
[2022-11-29T01:37:08,700][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [repository-s3]
[2022-11-29T01:37:08,700][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [repository-url]
[2022-11-29T01:37:08,700][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [runtime-fields-common]
[2022-11-29T01:37:08,700][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [search-business-rules]
[2022-11-29T01:37:08,700][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [searchable-snapshots]
[2022-11-29T01:37:08,701][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [snapshot-based-recoveries]
[2022-11-29T01:37:08,701][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [snapshot-repo-test-kit]
[2022-11-29T01:37:08,701][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [spatial]
[2022-11-29T01:37:08,701][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [transform]
[2022-11-29T01:37:08,701][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [transport-netty4]
[2022-11-29T01:37:08,701][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [unsigned-long]
[2022-11-29T01:37:08,701][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [vector-tile]
[2022-11-29T01:37:08,702][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [wildcard]
[2022-11-29T01:37:08,702][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-aggregate-metric]
[2022-11-29T01:37:08,702][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-analytics]
[2022-11-29T01:37:08,702][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-async]
[2022-11-29T01:37:08,702][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-async-search]
[2022-11-29T01:37:08,702][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-autoscaling]
[2022-11-29T01:37:08,702][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ccr]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-core]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-deprecation]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-enrich]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-eql]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-fleet]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-graph]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-identity-provider]
[2022-11-29T01:37:08,703][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ilm]
[2022-11-29T01:37:08,704][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-logstash]
[2022-11-29T01:37:08,704][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ml]
[2022-11-29T01:37:08,704][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-monitoring]
[2022-11-29T01:37:08,704][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-ql]
[2022-11-29T01:37:08,704][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-rollup]
[2022-11-29T01:37:08,704][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-security]
[2022-11-29T01:37:08,704][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-shutdown]
[2022-11-29T01:37:08,705][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-sql]
[2022-11-29T01:37:08,705][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-stack]
[2022-11-29T01:37:08,705][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-text-structure]
[2022-11-29T01:37:08,705][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-voting-only-node]
[2022-11-29T01:37:08,705][INFO ][o.e.p.PluginsService     ] [node-1] loaded module [x-pack-watcher]
[2022-11-29T01:37:08,705][INFO ][o.e.p.PluginsService     ] [node-1] no plugins loaded
[2022-11-29T01:37:10,649][WARN ][stderr                   ] [node-1] Nov 29, 2022 1:37:10 AM org.apache.lucene.store.MMapDirectory lookupProvider
[2022-11-29T01:37:10,649][WARN ][stderr                   ] [node-1] WARNING: You are running with Java 19. To make full use of MMapDirectory, please pass '--enable-preview' to the Java command line.
[2022-11-29T01:37:10,657][INFO ][o.e.e.NodeEnvironment    ] [node-1] using [1] data paths, mounts [[/data/openalex (/dev/vdb1)]], net usable_space [514.7gb], net total_space [934.9gb], types [ext4]
[2022-11-29T01:37:10,658][INFO ][o.e.e.NodeEnvironment    ] [node-1] heap size [31gb], compressed ordinary object pointers [true]
[2022-11-29T01:37:10,724][INFO ][o.e.n.Node               ] [node-1] node name [node-1], node ID [0bEGr_eeRMqPxoo6Sb7HtQ], cluster name [IShare], roles [data_cold, ingest, data_frozen, ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data]
[2022-11-29T01:37:13,499][INFO ][o.e.x.s.Security         ] [node-1] Security is enabled
[2022-11-29T01:37:13,794][INFO ][o.e.x.s.a.s.FileRolesStore] [node-1] parsed [0] roles from file [/usr/local/elasticsearch-8.5.1/config/roles.yml]
[2022-11-29T01:37:14,213][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [node-1] [controller/2338] [Main.cc@123] controller (64 bit): Version 8.5.1 (Build ac5909e3891a6b) Copyright (c) 2022 Elasticsearch BV
[2022-11-29T01:37:14,689][INFO ][o.e.t.n.NettyAllocator   ] [node-1] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=16mb}]
[2022-11-29T01:37:14,710][INFO ][o.e.i.r.RecoverySettings ] [node-1] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2022-11-29T01:37:14,742][INFO ][o.e.d.DiscoveryModule    ] [node-1] using discovery type [multi-node] and seed hosts providers [settings]
[2022-11-29T01:37:15,724][INFO ][o.e.n.Node               ] [node-1] initialized
[2022-11-29T01:37:15,725][INFO ][o.e.n.Node               ] [node-1] starting ...
[2022-11-29T01:37:15,739][INFO ][o.e.x.s.c.f.PersistentCache] [node-1] persistent cache index loaded
[2022-11-29T01:37:15,739][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [node-1] deprecation component started
[2022-11-29T01:37:15,817][INFO ][o.e.t.TransportService   ] [node-1] publish_address {192.168.0.240:9300}, bound_addresses {[::]:9300}
[2022-11-29T01:37:16,181][INFO ][o.e.b.BootstrapChecks    ] [node-1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-11-29T01:37:16,205][INFO ][o.e.n.Node               ] [node-1] stopping ...
[2022-11-29T01:37:16,252][INFO ][o.e.n.Node               ] [node-1] stopped
[2022-11-29T01:37:16,252][INFO ][o.e.n.Node               ] [node-1] closing ...
[2022-11-29T01:37:16,260][INFO ][o.e.n.Node               ] [node-1] closed
[2022-11-29T01:37:16,261][INFO ][o.e.x.m.p.NativeController] [node-1] Native controller process has stopped - no new native processes can be started
[2022-11-29T01:37:16,262][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [process reaper (pid 2338)]
java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "modifyThread")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:485) ~[?:?]
	at java.security.AccessController.checkPermission(AccessController.java:1068) ~[?:?]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:411) ~[?:?]
	at org.elasticsearch.secure_sm.SecureSM.checkThreadAccess(SecureSM.java:166) ~[?:?]
	at org.elasticsearch.secure_sm.SecureSM.checkAccess(SecureSM.java:120) ~[?:?]
	at java.lang.Thread.checkAccess(Thread.java:2360) ~[?:?]
	at java.lang.Thread.setDaemon(Thread.java:2308) ~[?:?]
	at java.lang.ProcessHandleImpl.lambda$static$0(ProcessHandleImpl.java:103) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:637) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:928) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.processWorkerExit(ThreadPoolExecutor.java:1021) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1158) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1589) ~[?:?]
	at jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:186) ~[?:?]

warkolm · November 29, 2022, 2:13am

Welcome to our community!

hrQAQ:

[2022-11-29T01:37:15,817][INFO ][o.e.t.TransportService   ] [node-1] publish_address {192.168.0.240:9300}, bound_addresses {[::]:9300}
[2022-11-29T01:37:16,181][INFO ][o.e.b.BootstrapChecks    ] [node-1] bound or publishing to a non-loopback address, enforcing bootstrap checks

This says that Elasticsearch has started ok.

And this says that something is telling Elasticsearch to shutdown before it can bind the HTTP port.

What are the permissions on these directories?

hrQAQ · November 29, 2022, 7:36am

Thx so much for your reply !

I'v solved my problem already.

When I looked at the error printed on the console(not the log of elasticsearch) when starting elastic, I found that it told me bootstrap failed because I did not turn on xpack.security.transport.ssl.enabled: true(I don't remember the exact error text)

Though I didn't understand why I had to set it when network=0.0.0.0), I followed the instructions to enable xpack.security.transport.ssl.enabled: true, and then followed the official tutorial to finish the ssl-related configuration, finanally I was able to run it very well.

Anyway, Thx again for your reply !

hrQAQ · November 29, 2022, 7:43am

By the way, I also change the security manager in JRE（which is bound with elasticsearch）, add

grant{
  // other deault configurations
  ...
  // I add this permission
  java.lang.RuntimePermission" "modifyThread"
}

I don't know whether it is neccessary to do this step, This is for reference only

system · December 27, 2022, 7:44am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.