License 2.1.1 Jar Hell Error

Elastic Stack Elasticsearch
1

Following is the error I am seeing on my nodes :

[2016-03-17 17:37:22,708][ERROR][bootstrap ] Exception
java.lang.IllegalStateException: failed to load bundle [file:/D:/ElasticSearch/pluginVersioned/license/2.2.1/license-2.2.1.jar, file:/D:/ElasticSearch/pluginVersioned/license/2.2.1/license-core-2.2.1.jar, file:/D:/ElasticSearch/pluginVersioned/license/2.2.1/license-plugin-api-2.2.1.jar, file:/D:/ElasticSearch/pluginVersioned/marvel-agent/2.2.1/marvel-agent-2.2.1.jar] due to jar hell
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:421)
at org.elasticsearch.plugins.PluginsService.(PluginsService.java:129)
at org.elasticsearch.node.Node.(Node.java:146)
at org.elasticsearch.node.Node.(Node.java:128)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:285)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "D:\ElasticSearch\pluginVersioned\license\2.2.1\license-2.2.1.jar" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
at java.security.AccessController.checkPermission(AccessController.java:559)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.util.zip.ZipFile.(ZipFile.java:206)
at java.util.zip.ZipFile.(ZipFile.java:145)
at java.util.jar.JarFile.(JarFile.java:153)
at java.util.jar.JarFile.(JarFile.java:90)
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:174)
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:419)
... 7 more

I don't want to disable the Java Security Manager. I have added a plugin-security.policy file in the License plugin folder next to the jars. Here's what the file looks like :

grant {
// needed to set expert SSL options, etc
permission java.lang.RuntimePermission "setFactory";
permission javax.net.ssl.SSLPermission "setHostnameVerifier";
permission java.io.FilePermission "*";
};

Even then I get the above error.

2

How did you install Elasticsearch? Did you change anything from the default configuration?

I don't recognize the folder "pluginVersioned" - is that something you created?

Please do not change or add to the contents of the plugin directory - it is not necessary, and we need to figure out what the real problem is.

3

elasticsearch is installed using offline method ... i.e. download it, change the yml file, and point elasticsearch to that yml file using the config parameter.

The only change I made was the bin/elasticsearch.bat file to add another argument for picking the config from another location.

pluginVersioned is another folder where I store all my plugins and initializing from there. There is no other plugin directory.

4

Can you try this in a fresh install, with default settings and directories? It sounds like you made a lot of changes to the configuration, and they are clearly the cause of the issue.

Are you installing the plugin using bin/plugin?

5

Fresh install works fine when the plugins are directly under the plugins folder next to bin folder. The problem is when using the path.plugins parameter in elasticsearch.yml file point to a different location. Additional complication in my case was that the plugins folder was symlinked.

Since the path.plugins parameter is still supported, can I get an example of how to setup the java security policy in the plugin folders if the folders are not under the elasticsearch folder ?

6

IIRC we don't support symlinks for some dirs (if not all) so it's probably the issue here.

7

Can you explain your use-case for needing to specify path.plugins?

8

symlinks definitely worked before 2.0.0 .. we have been using it ... did it change after > 2.0.0 ?

9

We have a separate folder structure where we have all of our plugins for each version. Similarly for elasticsearch and kibana. Then we have a deployment solution which allows us to deploy the desired versions of elasticsearch and kibana along with desired plugins by simply symlinking them under one folder and then using path.plugins. This allows us to deploy different configurations on different environments.

However, now, marvel and license strictly have to be of the same version as elasticsearch. This slightly makes things simpler, coz there is no way to configure different versions of these plugins for elasticsearch. Hence, I directly pushed these plugins under the elasticsearch plugins folder. But this doesn't allow me now to "not deploy" the plugins in another environment. And for other plugins which may not have strict requirements as marvel and license, this problem still stands.

10

Yes it did.