# Custom Realm and JarHell **URL:** \ **Category:** Elasticsearch\ **Created:** [April 12, 2017, 3:53pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188 "2017-04-12T15:53:19Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![jhn134910](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jhn134910](https://discuss.elastic.co/u/jhn134910)\ **Post date:** [April 12, 2017, 3:53pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/1 "2017-04-12T15:53:19Z")
Hello there, I've attempted to author a custom realm as a fat JAR. When installing it I get the following JarHell error: [root@iel-dev-tfsr-vm1 x-pack]# ./extension install file:///home/tango/john/ims-realm-1.0.0-SNAPSHOT.zip -\> Downloading file:///home/tango/john/ims-realm-1.0.0-SNAPSHOT.zip Exception in thread "main" java.lang.IllegalStateException: jar hell! class: org.apache.commons.logging.impl.AvalonLogger jar1: /usr/share/elasticsearch/plugins/x-pack/commons-logging-1.1.3.jar jar2: /usr/share/elasticsearch/plugins/x-pack/extensions/.installing-1737085689021563353/ims-realm-1.0.0-SNAPSHOT.jar at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:277) at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:187) at org.elasticsearch.xpack.extensions.InstallXPackExtensionCommand.jarHellCheck(InstallXPackExtensionCommand.java:197) at org.elasticsearch.xpack.extensions.InstallXPackExtensionCommand.verify(InstallXPackExtensionCommand.java:169) at org.elasticsearch.xpack.extensions.InstallXPackExtensionCommand.install(InstallXPackExtensionCommand.java:207) at org.elasticsearch.xpack.extensions.InstallXPackExtensionCommand.execute(InstallXPackExtensionCommand.java:116) at org.elasticsearch.xpack.extensions.InstallXPackExtensionCommand.execute(InstallXPackExtensionCommand.java:102) at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:69) at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) at org.elasticsearch.cli.Command.main(Command.java:88) at org.elasticsearch.xpack.extensions.XPackExtensionCli.main(XPackExtensionCli.java:36) Do you know how to trouble-shoot or track down the offending dependency? I have quite a few dependencies: org.elasticsearch elasticsearch 5.1.2 provided org.elasticsearch.plugin x-pack-api 5.1.2 provided net.java.dev.jna jna 4.2.2 provided org.springframework spring-core 4.3.2.RELEASE compile org.springframework spring-web 4.3.2.RELEASE compile org.springframework.security spring-security-core 4.2.1.RELEASE provided org.apache.logging.log4j log4j-api 2.7 compile org.apache.logging.log4j log4j-core 2.7 compile org.meanbean meanbean 2.0.3 test nl.jqno.equalsverifier equalsverifier 1.7.7 test org.projectlombok lombok 1.14.8 compile com.google.guava guava 18.0 compile com.tango.common.utils tango-common-utils 1.1.0 compile com.tango.common.mvc tango-common-mvc 1.1.3-SNAPSHOT compile com.tango.ims ims-client 2.2.4-SNAPSHOT compile junit junit 4.12 test org.mockito mockito-core 1.10.19 test //John. ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [April 12, 2017, 11:28pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/2 "2017-04-12T23:28:46Z")
```auto Exception in thread "main" java.lang.IllegalStateException: jar hell! class: org.apache.commons.logging.impl.AvalonLogger jar1: /usr/share/elasticsearch/plugins/x-pack/commons-logging-1.1.3.jar jar2: /usr/share/elasticsearch/plugins/x-pack/extensions/.installing-1737085689021563353/ims-realm-1.0.0-SNAPSHOT.jar ``` Somewhere in your chain of dependencies you are including `commons-logging` and then repackaging it into your uberjar. The best way to fixing that is going depend on what your build process looks like. If you're using maven, then ``` mvn dependency:tree ``` will show you all your dependencies and how they're being included. Then you could add an explicit exclusion on commons-logging. Alternatively it might be easier to do it at the stage where you build the uberjar. Just drop any dependencies that are part of x-pack. ---
**Author:** ![jhn134910](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jhn134910](https://discuss.elastic.co/u/jhn134910)\ **Post date:** [April 13, 2017, 8:25am UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/3 "2017-04-13T08:25:57Z")
Thank you Tim - I've followed your guidance and have successfully installed my custom realm. My custom realm attempts to authenticate against an external system by sending a HTTP request to that system. On initialization it attempts to create the HTTP client. However, I'm receiving a permissions problem: Caused by: java.security.AccessControlException: access denied ("java.net.NetPermission" "getProxySelector") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0\_92] at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0\_92] at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0\_92] at java.net.ProxySelector.getDefault(ProxySelector.java:94) ~[?:1.8.0\_92] at org.apache.http.impl.client.HttpClientBuilder.build(HttpClientBuilder.java:1147) ~[?:?] at org.apache.http.impl.client.HttpClients.createSystem(HttpClients.java:66) ~[?:?] at org.springframework.http.client.HttpComponentsClientHttpRequestFactory.(HttpComponentsClientHttpRequestFactory.java:88) ~[?:?] at org.springframework.http.client.HttpComponentsAsyncClientHttpRequestFactory.(HttpComponentsAsyncClientHttpRequestFactory.java:67) ~[?:?] at com.tango.common.mvc.client.AsyncClient.(AsyncClient.java:61) ~[?:?] at com.tango.common.mvc.client.AsyncClient.(AsyncClient.java:45) ~[?:?] at com.tango.common.mvc.client.AsyncClient$Builder.build(AsyncClient.java:267) ~[?:?] at com.tango.xpack.realm.IMSRealm.(IMSRealm.java:67) ~[?:?] at com.tango.xpack.realm.IMSRealmFactory.create(IMSRealmFactory.java:19) ~[?:?] at com.tango.xpack.realm.IMSRealmFactory.create(IMSRealmFactory.java:11) ~[?:?] at org.elasticsearch.xpack.security.authc.Realms.initRealms(Realms.java:182) ~[?:?] at org.elasticsearch.xpack.security.authc.Realms.doStart(Realms.java:82) ~[?:?] at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.1.2.jar:5.1.2] at java.util.ArrayList.forEach(ArrayList.java:1249) ~[?:1.8.0\_92] at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080) ~[?:1.8.0\_92] at org.elasticsearch.node.Node.start(Node.java:542) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:234) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:308) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.2.jar:5.1.2] I attempted to add a plugin-security.policy file on my classpath with the following grant - permission java.net.NetPermission "\*"; However, I get the same result? Any ideas how to resolve this one? ---
**Author:** ![jhn134910](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jhn134910](https://discuss.elastic.co/u/jhn134910)\ **Post date:** [April 13, 2017, 2:24pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/4 "2017-04-13T14:24:45Z")
I see somebody else had the same issue: > [@How to customize plugin-security.policy for custom realm](https://discuss.elastic.co/t/how-to-customize-plugin-security-policy-for-custom-realm/71570/5): > > So, now, after installing the custom realm extension I have a file named x-pack-extension-security.policy in /usr/share/elasticsearch/plugins/x-pack/extensions/mycustomrealm. The contents of the file is: grant { permission java.net.NetPermission "getProxySelector"; }; I am still getting an error: java.security.AccessControlException: access denied ("java.net.NetPermission" "getProxySelector") when the client is instantiated. Any ideas? This is with 5.1.1. I followed the steps suggested there. Unfortunately, though it doesn't seem to be picking up the permission I specified in x-pack-extension-security.policy. I add code like: if(sm != null) { sm.checkPermission(new NetworkPermission("getProxySelector")); } AccessController.doPrivileged((PrivilegedAction) () -\> { final AsyncClient asyncClient = new AsyncClient.Builder() .withConnectionTimeout(connectionTimeout).withSocketTimeout(socketTimeout) .withMaxConnections(maxConnections).withMaxConnectionsPerHost(maxConnectionsPerHost).build(); ``` this.imsClient = new IMSClient("http://localhost", Optional.of(asyncClient), imsCacheSize, imsCacheExpiryInMinutes); return null; ``` }); However, the checkPermission is saying I don't have permission even after the I okayed the permissions when installing the custom realm plugin and the .policy file is present [root@iel-dev-tfsr-vm1 tango-ims-realm]# pwd /usr/share/elasticsearch/plugins/x-pack/extensions/tango-ims-realm [root@iel-dev-tfsr-vm1 tango-ims-realm]# ls ims-realm-1.0.0-SNAPSHOT.jar x-pack-extension-descriptor.properties x-pack-extension-security.policy ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [April 13, 2017, 4:39pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/5 "2017-04-13T16:39:38Z")
Can you share the full contents of the policy file? Is it the same exact exception? ---
**Author:** ![jhn134910](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jhn134910](https://discuss.elastic.co/u/jhn134910)\ **Post date:** [April 14, 2017, 8:00am UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/6 "2017-04-14T08:00:40Z")
Thanks for your response. The contents of my x-pack-extension-descriptor.properties: description=Tango IMS Realm Extension version=1.0.0-SNAPSHOT name=tango-ims-realm classname=com.tango.xpack.IMSRealmExtension java.version=1.8 xpack.version=5.1.2 The contents of my x-pack-extension-security.policy: grant { permission java.net.NetPermission "getProxySelector"; }; I'm building my fat JAR with maven assembly plugin. Then when attempting to install I do the following: [root@iel-dev-tfsr-vm1 x-pack]# ./extension remove tango-ims-realm -\> Removing tango-ims-realm... [root@iel-dev-tfsr-vm1 x-pack]# ./extension install file:///home/tango/john/tango-ims-realm-1.0.0-SNAPSHOT.zip -\> Downloading file:///home/tango/john/tango-ims-realm-1.0.0-SNAPSHOT.zip @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: x-pack extension requires additional permissions @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ - java.net.NetPermission getProxySelector See [http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html](http://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html) for descriptions of what these permissions allow and the associated risks. Continue with installation? [y/N]y -\> Installed tango-ims-realm [root@iel-dev-tfsr-vm1 x-pack]# pwd /usr/share/elasticsearch/bin/x-pack [root@iel-dev-tfsr-vm1 x-pack]# cd ../../plugins/x-pack/extensions/;chmod 755 \* [root@iel-dev-tfsr-vm1 extensions]# !1667 chown -R elasticsearch:elasticsearch /usr/share/elasticsearch && cd /var/log/elasticsearch/ && tail -f tango-cluster.log [2017-04-14T08:53:05,484][INFO][o.e.n.Node] [] initializing ... [2017-04-14T08:53:05,554][INFO][o.e.e.NodeEnvironment] [vCUgnmd] using [1] data paths, mounts [[/ (rootfs)]], net usable\_space [2.8gb], net total\_space [18.4gb], spins? [unknown], types [rootfs] [2017-04-14T08:53:05,554][INFO][o.e.e.NodeEnvironment] [vCUgnmd] heap size [1.9gb], compressed ordinary object pointers [true] [2017-04-14T08:53:05,599][INFO][o.e.n.Node] node name [vCUgnmd] derived from node ID [vCUgnmdpTsevehNdEv9E5A]; set [[node.name](http://node.name)] to override [2017-04-14T08:53:05,601][INFO][o.e.n.Node] version[5.1.2], pid[36275], build[c8c4c16/2017-01-11T20:18:39.146Z], OS[Linux/3.10.0-229.20.1.el7.x86\_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0\_92/25.92-b14] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [aggs-matrix-stats] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [ingest-common] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [lang-expression] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [lang-groovy] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [lang-mustache] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [lang-painless] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [percolator] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [reindex] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [transport-netty3] [2017-04-14T08:53:07,077][INFO][o.e.p.PluginsService] [vCUgnmd] loaded module [transport-netty4] [2017-04-14T08:53:07,078][INFO][o.e.p.PluginsService] [vCUgnmd] loaded plugin [x-pack] [2017-04-14T08:53:09,624][INFO][o.e.n.Node] initialized [2017-04-14T08:53:09,625][INFO][o.e.n.Node] [vCUgnmd] starting ... [2017-04-14T08:53:09,628][ERROR][o.e.b.Bootstrap] [vCUgnmd] Exception ... org.elasticsearch.bootstrap.StartupException: java.security.AccessControlException: access denied ("jdk.net.NetworkPermission" "getProxySelector") at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.cli.Command.main(Command.java:88) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.1.2.jar:5.1.2] Caused by: java.security.AccessControlException: access denied ("jdk.net.NetworkPermission" "getProxySelector") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0\_92] at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0\_92] at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0\_92] at com.tango.xpack.realm.IMSRealm.(IMSRealm.java:69) ~[?:?] at com.tango.xpack.realm.IMSRealmFactory.create(IMSRealmFactory.java:19) ~[?:?] at com.tango.xpack.realm.IMSRealmFactory.create(IMSRealmFactory.java:11) ~[?:?] at org.elasticsearch.xpack.security.authc.Realms.initRealms(Realms.java:182) ~[?:?] at org.elasticsearch.xpack.security.authc.Realms.doStart(Realms.java:82) ~[?:?] at org.elasticsearch.common.component.AbstractLifecycleComponent.start(AbstractLifecycleComponent.java:69) ~[elasticsearch-5.1.2.jar:5.1.2] at java.util.ArrayList.forEach(ArrayList.java:1249) ~[?:1.8.0\_92] at java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1080) ~[?:1.8.0\_92] at org.elasticsearch.node.Node.start(Node.java:542) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Bootstrap.start(Bootstrap.java:234) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:308) ~[elasticsearch-5.1.2.jar:5.1.2] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.2.jar:5.1.2] ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [April 14, 2017, 11:21am UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/7 "2017-04-14T11:21:33Z")
It looks like it might boil down to just an incorrect permission name: > [@jhn134910](#): > > The contents of my x-pack-extension-security.policy:grant { permission java.net.NetPermission "getProxySelector";}; > [@jhn134910](#): > > Caused by: java.security.AccessControlException: access denied ("jdk.net.NetworkPermission" "getProxySelector") at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0\_92] at java.security.AccessController.checkPermission(AccessController.java:884) ~[?:1.8.0\_92] at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0\_92] at com.tango.xpack.realm.IMSRealm.(IMSRealm.java:69) ~[?:?] Try using `jdk.net.NetworkPermission` instead of `java.net.NetPermission` in your policy file. ---
**Author:** ![jhn134910](https://avatars.discourse-cdn.com/v4/letter/j/90ced4/32.png) [@jhn134910](https://discuss.elastic.co/u/jhn134910)\ **Post date:** [April 14, 2017, 12:59pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/8 "2017-04-14T12:59:37Z")
Jeez - I should have spotted that. Staring at it too long. This mismatch was caused by my coding: if(sm != null) { sm.checkPermission(new NetworkPermission("getProxySelector")); } When I changed it to the correct if(sm != null) { sm.checkPermission(new NetPermission("getProxySelector")); } then the exception lines up with what I have in my grant file. When I add the line permission java.net.NetPermission "getProxySelector"; to the file: /usr/share/elasticsearch/plugins/x-pack/plugin-security.policy then is seems to work, but I guess this is giving the permission right across x-pack. Should my policy file also be called plugin-security.policy? ---
**Author:** ![jaymode](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jaymode/32/50103_2.png) [@jaymode](https://discuss.elastic.co/u/jaymode)\ **Post date:** [April 14, 2017, 2:20pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/9 "2017-04-14T14:20:35Z")
> [@jhn134910](#): > > but I guess this is giving the permission right across x-pack. This is only granted to your extension. > [@jhn134910](#): > > Should my policy file also be called plugin-security.policy No, `x-pack-extension-security.policy` is the correct name. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 12, 2017, 2:34pm UTC](https://discuss.elastic.co/t/custom-realm-and-jarhell/82188/10 "2017-05-12T14:34:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.