How to grant permission in ESClientYamlSuiteTestCase integTest?

Hello,
I wrote a plugin which provide support for Datasketches aggregation, I think it is easy to use and I want to contribute it.
I added it to the plugins folder with code, unit tests and all license for dependencies.
My question is, when i'm running my ESClientYamlSuiteTestCase for integTest, It fails because of java security policy problem. Since My plugin use Apache Datasketches, which needs to access sun.misc , I need grant permission on accessDeclaredMembers, accessClassInPackage.sun.misc and accessClassInPackage.sun.misc.*,etc.
I need grant those permission in integTest environment, so I make a temporary folder by gradle script:

def policyPath = file("${buildDir}/tmp")
if (!policyPath.exists()) {
mkdir policyPath
}

I write all permission I needed in a policy file:

File policyFile = file("${buildDir}/tmp/java.policy")
policyFile.write(
  [
    "grant {",
    "permission java.lang.RuntimePermission \"accessDeclaredMembers\";",
    "permission java.lang.RuntimePermission \"accessClassInPackage.sun.misc\";",
    "permission java.lang.RuntimePermission \"accessClassInPackage.sun.misc.*\";",
    "permission java.lang.reflect.ReflectPermission \"suppressAccessChecks\";",
    "};"
  ].join("\n")
)

Combine them in a task called writeJavaPolicy, and make sure it will run before integTestRunner:

integTest.runner {
  dependsOn 'writeJavaPolicy'
  jvmArgs "-Djava.security.policy=${buildDir}/tmp/java.policy"
  systemProperty 'java.security.policy', "file://${buildDir}/tmp/java.policy"
}

I run the integTestRunner task, and it also failed because of the permission problem.
I can't tell whats wrong with that task. So I decided turn the security.manager off temporary:

integTest.runner {
  systemProperty 'tests.security.manager', false
}

turned security.manager off and rerun the task, it failed again, with same error, even thought the security.manager turned off.
Here is the node log and permission error looks like:

[2020-07-21T15:02:45,530][INFO ][o.e.n.Node               ] [integTest-0] version[8.0.0-SNAPSHOT], pid[50692], build[oss/zip/eecef2a6ed6b3186939beb96fa35857f315dc9ce/2020-06-28T11:05:20.560252Z], OS[Mac OS X/10.15.5/x86_64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/14.0.1/14.0.1+7]
[2020-07-21T15:02:45,535][INFO ][o.e.n.Node               ] [integTest-0] JVM home [/Users/testhaha/Library/Java/JavaVirtualMachines/openjdk-14.0.1/Contents/Home]
[2020-07-21T15:02:45,535][INFO ][o.e.n.Node               ] [integTest-0] JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPercent=30, -Djava.io.tmpdir=/Users/testhaha/Documents/git/elasticsearch/plugins/search-datasketches/build/testclusters/integTest-0/tmp, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Xms512m, -Xmx512m, -ea, -esa, -XX:MaxDirectMemorySize=268435456, -Des.path.home=/Users/testhaha/Documents/git/elasticsearch/plugins/search-datasketches/build/testclusters/integTest-0/distro/8.0.0-INTEG_TEST, -Des.path.conf=/Users/testhaha/Documents/git/elasticsearch/plugins/search-datasketches/build/testclusters/integTest-0/config, -Des.distribution.flavor=oss, -Des.distribution.type=zip, -Des.bundled_jdk=false]
[2020-07-21T15:02:45,536][WARN ][o.e.n.Node               ] [integTest-0] version [8.0.0-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production
[2020-07-21T15:02:45,801][INFO ][o.e.p.PluginsService     ] [integTest-0] loaded module [transport-netty4]
[2020-07-21T15:02:45,801][INFO ][o.e.p.PluginsService     ] [integTest-0] loaded plugin [search-datasketches]
[2020-07-21T15:02:45,848][INFO ][o.e.e.NodeEnvironment    ] [integTest-0] using [1] data paths, mounts [[/System/Volumes/Data (/dev/disk1s1)]], net usable_space [411.8gb], net total_space [465.6gb], types [apfs]
[2020-07-21T15:02:45,849][INFO ][o.e.e.NodeEnvironment    ] [integTest-0] heap size [512mb], compressed ordinary object pointers [true]
[2020-07-21T15:02:45,886][INFO ][o.e.n.Node               ] [integTest-0] node name [integTest-0], node ID [8yKT_kjOTUyo6uanKRQ77A], cluster name [integTest]
[2020-07-21T15:02:46,892][INFO ][o.e.d.DiscoveryModule    ] [integTest-0] using discovery type [zen] and seed hosts providers [settings, file]
[2020-07-21T15:02:47,177][INFO ][o.e.n.Node               ] [integTest-0] initialized
[2020-07-21T15:02:47,177][INFO ][o.e.n.Node               ] [integTest-0] starting ...
[2020-07-21T15:02:47,317][INFO ][o.e.t.TransportService   ] [integTest-0] publish_address {127.0.0.1:49826}, bound_addresses {[::1]:49825}, {127.0.0.1:49826}
[2020-07-21T15:02:47,519][WARN ][o.e.d.FileBasedSeedHostsProvider] [integTest-0] expected, but did not find, a dynamic hosts list at [/Users/testhaha/Documents/git/elasticsearch/plugins/search-datasketches/build/testclusters/integTest-0/config/unicast_hosts.txt]
[2020-07-21T15:02:47,519][INFO ][o.e.c.c.Coordinator      ] [integTest-0] setting initial configuration to VotingConfiguration{8yKT_kjOTUyo6uanKRQ77A}
[2020-07-21T15:02:47,593][INFO ][o.e.h.AbstractHttpServerTransport] [integTest-0] publish_address {127.0.0.1:49828}, bound_addresses {[::1]:49827}, {127.0.0.1:49828}
[2020-07-21T15:02:47,595][INFO ][o.e.n.Node               ] [integTest-0] started
[2020-07-21T15:02:47,633][INFO ][o.e.c.s.MasterService    ] [integTest-0] elected-as-master ([1] nodes joined)[{integTest-0}{8yKT_kjOTUyo6uanKRQ77A}{pWq7aAJFSSOReNqTkcWpmw}{127.0.0.1}{127.0.0.1:49826}{dimr}{testattr=test} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 1, version: 1, delta: master node changed {previous [], current [{integTest-0}{8yKT_kjOTUyo6uanKRQ77A}{pWq7aAJFSSOReNqTkcWpmw}{127.0.0.1}{127.0.0.1:49826}{dimr}{testattr=test}]}
[2020-07-21T15:02:47,681][INFO ][o.e.c.c.CoordinationState] [integTest-0] cluster UUID set to [20SyT1fyTvSOchej3QY9-g]
[2020-07-21T15:02:47,717][INFO ][o.e.c.s.ClusterApplierService] [integTest-0] master node changed {previous [], current [{integTest-0}{8yKT_kjOTUyo6uanKRQ77A}{pWq7aAJFSSOReNqTkcWpmw}{127.0.0.1}{127.0.0.1:49826}{dimr}{testattr=test}]}, term: 1, version: 1, reason: Publication{term=1, version=1}
[2020-07-21T15:02:47,762][INFO ][o.e.g.GatewayService     ] [integTest-0] recovered [0] indices into cluster_state
[2020-07-21T15:02:50,981][INFO ][o.e.c.m.MetadataCreateIndexService] [integTest-0] [test] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[2020-07-21T15:02:51,580][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [integTest-0] fatal error in thread [elasticsearch[integTest-0][search][T#1]], exiting
java.lang.ExceptionInInitializerError: null
	at org.apache.datasketches.memory.Memory.wrap(Memory.java:277) ~[?:?]
	at org.apache.datasketches.memory.Memory.wrap(Memory.java:244) ~[?:?]
	at org.apache.datasketches.hll.Union.heapify(Union.java:105) ~[?:?]
	at org.elasticsearch.search.aggregations.metrics.datasketches.hll.HllSketchAggregator$1.collect(HllSketchAggregator.java:139) ~[?:?]
	at org.elasticsearch.search.aggregations.LeafBucketCollector.collect(LeafBucketCollector.java:107) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.apache.lucene.search.MultiCollector$MultiLeafCollector.collect(MultiCollector.java:188) ~[lucene-core-8.6.0-snapshot-e7c625430ed.jar:8.6.0-snapshot-e7c625430ed e7c625430ed916fd7d0e7e8d1cdc329093d817b4 - jenkins - 2020-06-11 10:26:00]
	at org.apache.lucene.search.MatchAllDocsQuery$1$1.score(MatchAllDocsQuery.java:64) ~[lucene-core-8.6.0-snapshot-e7c625430ed.jar:8.6.0-snapshot-e7c625430ed e7c625430ed916fd7d0e7e8d1cdc329093d817b4 - jenkins - 2020-06-11 10:26:00]
	at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:56) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.6.0-snapshot-e7c625430ed.jar:8.6.0-snapshot-e7c625430ed e7c625430ed916fd7d0e7e8d1cdc329093d817b4 - jenkins - 2020-06-11 10:26:00]
	at org.elasticsearch.search.internal.ContextIndexSearcher.searchLeaf(ContextIndexSearcher.java:213) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:186) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445) ~[lucene-core-8.6.0-snapshot-e7c625430ed.jar:8.6.0-snapshot-e7c625430ed e7c625430ed916fd7d0e7e8d1cdc329093d817b4 - jenkins - 2020-06-11 10:26:00]
	at org.elasticsearch.search.query.QueryPhase.searchWithCollector(QueryPhase.java:347) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.search.query.QueryPhase.executeInternal(QueryPhase.java:298) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:150) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:362) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:435) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.search.SearchService$2.lambda$onResponse$0(SearchService.java:396) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.search.SearchService.lambda$runAsync$0(SearchService.java:412) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:706) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-8.0.0-SNAPSHOT.jar:8.0.0-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) ~[?:?]
	at java.lang.Thread.run(Thread.java:832) [?:?]
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "accessClassInPackage.sun.misc")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
	at java.security.AccessController.checkPermission(AccessController.java:1036) ~[?:?]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:408) ~[?:?]
	at java.lang.SecurityManager.checkPackageAccess(SecurityManager.java:1376) ~[?:?]
	at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:852) ~[?:?]
	at java.lang.ClassLoader.loadClass(ClassLoader.java:522) ~[?:?]
	at org.apache.datasketches.memory.UnsafeUtil.<clinit>(UnsafeUtil.java:87) ~[?:?]
	... 25 more
[2020-07-21T07:02:52.205594Z] [BUILD] Stopping node

It is easy to find out its jvm option is not contained security policy settings, SO what is the real way to set jvm security settings for integTest?
Skip IntegTest is not an option, plugins in elasticsearch project all have runable IntegTest. after all, every commits should come with a good IntegTest.
So Please help me, Thank you very much.

Hello,Is anyone has an idea?

So, if your plugin code is needing those privileges, then you have to add those permissions in the src/main/plugin-metadata/plugin-security.policy - this is the file used on plugin loading to allow permissions like the above one.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.