ElasticSearch 2.0 java api JarHell problems

Hello everybody,

I'm trying to migrate from elasticsearch 1.6.0 to 2.0.0 using the Java API. The tests were passing with 1.60. However, after I made the tests compile with 2.0.0 my tests are failing when running them locally due to various exceptions generated by JarHell. The question is, can I disable the JarHell check for local testing?

Thanks!

If you are running your tests from IntelliJ, that can be the cause.
Otherwise, I would try to fix any JAR conflict in my project instead of trying to disable JarHell check.

Hello,

Thanks for the response.
Actually , I'm running the tests using gradle 2.6, jdk 7 also tried it with 8.
Could you please tell me how could I work around such an issue ?

java.lang.RuntimeException: found jar hell in test classpath
Caused by:
java.lang.IllegalStateException: jar hell!
class: org.apache.hadoop.yarn.factories.package-info
jar1: /home/george/.gradle/caches/modules-2/files-2.1/org.apache.hadoop/hadoop-yarn-api/2.5.0-cdh5.3.0/7542d2b9b89956291d8fb4294aab143adf4f5252/hadoop-yarn-api-2.5.0-cdh5.3.0.jar
jar2: /home/george/.gradle/caches/modules-2/files-2.1/org.apache.hadoop/hadoop-yarn-common/2.5.0-cdh5.3.0/ab28ad57af245463f66a5da314dffc684fe9cc13/hadoop-yarn-common-2.5.0-cdh5.3.0.jar

This is about the package-info that happens to live in the same package in two different jars.

Thanks!

The "identical jar" is not a jar hell by Oracle's definition for a class path, but it was refused to fix: https://github.com/elastic/elasticsearch/issues/13604

So you have to do some very weird acrobatics in a gradle build, something like this for integration testing

task integrationTest(type: Test, dependsOn: ['unpackPlugin']) {
    testClassesDir = sourceSets.integrationTest.output.classesDir
    // the test dependencies including Elasticsearch jars
    classpath = configurations.integrationTestCompile
    // the dependencies of this plugin as jars from the plugin zip
    classpath += fileTree('plugins/<name>').include('*.jar')
    // our plugin classes and resources
    classpath += sourceSets.integrationTest.output
    // without removing identical jars from classpath, Elasticsearch whines about a "jar hell"
    classpath -= configurations.releaseJars
    outputs.upToDateWhen { false }
    systemProperty 'path.home', projectDir.absolutePath
}

The complete gradle build file is at https://github.com/jprante/elasticsearch-webapp/blob/master/build.gradle
It's a groovy plugin, but the 'identical jar' challenge also applies to Java plugin builds with gradle.

Hello,

But to be honest this JarHell is really restrictive. I mean take for example this package-info, which is actually not used at runtime. Why would you want to fail the build because of that? I should be able to decide as a client if this is fine with me or not. Elasticsearch should not decide about incompatibilities in other libraries in the same classpath, in my opinion.

@jprante Can you please suggest some way to run the tests locally and avoid JarHell? I have a simple test with no plugins, no nothing, only extending the ESIntegTestCase creating a mapping and retrievering it. I'm using the hadoop dependecy because I have a custom definition of the fields. Because I see that your gradle integrationTest depends on some unpackPlugin, which is not an option for me.

@George I never tried ESIntegTestCase, it looks too tricky to me

I see. I tried also once to play with Hadoop but it's really a JAR hell there!

Not sure about what you can do. I think @costin solved that issue on his end for the hadoop plugin. @costin how did you fix that?

Load Hadoop into its own ClassLoader so that it's not part of the ES/JarHell classpath.

Hi Costin,

Thanks for the response.
Can I do that in gradle when I run the test? Do you have some snippet or some example somewhere?

@dadoonet thanks for asking costin.
Thanks!

Not that I'm aware of - it's doable probably through some type of plugin however note that the JarHell can occur easily at runtime as well.
You can see the ClassLoader solution (which is Java Security Manager friendly as it does not require extra permissions) is here, in the repository-hdfs repo

I'm getting the same problem while running tests from Eclipse - running from maven works. Is there any solution to fix this issue in Eclipse ?

Caused by: java.lang.IllegalStateException: jar hell!
class: com...agent.restapi.transferobject.ClientRequestPopto
jar1: C:\WorkSoft\mvn-repo-security\com*
*
\2.5.00-RELEASE*-2.5.00-RELEASE.jar
jar2: C:\p4ws\ola*
**\target\classes
at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:267)

Fix your classpath. You can't have the same class twice in the classloader.

@dadoonet, I have checked the classpath - generate by m2e plugin- and it doesn't have any duplicate entries - classpath is generated my maven eclipse integration. And, I'm gettng this problem only while running tests from the Eclipse. Any clues on what is going wrong ?

Can you print the full log?

Here is the trace:

TestBWElasticSearchSink.testProcess_BulkRequestWithChunks1
Unrooted Tests
com.hidden.ola.collection.payload.TestBWElasticSearchSink
java.lang.IllegalStateException: failed to load bundle [] due to jar hell

	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:342)

	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:113)

	at org.elasticsearch.node.Node.<init>(Node.java:144)

	at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)

	at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:152)

	at com.hidden.ola.util.ElasticSearchTest.setUpBeforeClass(ElasticSearchTest.java:79)

	at com.hidden.ola.collection.payload.TestBWElasticSearchSink.setUpBeforeClass(TestBWElasticSearchSink.java:46)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.lang.reflect.Method.invoke(Method.java:497)

	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)

	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)

	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)

	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)

	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)

	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

Caused by: java.lang.IllegalStateException: jar hell!

class: com.hidden.ola.agent.restapi.transferobject.ClientRequestPopto

jar1: C:\WorkSoft\mvn-repo-security\com\hidden\ola\types\2.5.00-RELEASE\types-2.5.00-RELEASE.jar

jar2: C:\p4ws\ola\types\target\classes

	at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:267)

	at org.elasticsearch.bootstrap.JarHell$1.visitFile(JarHell.java:201)

	at org.elasticsearch.bootstrap.JarHell$1.visitFile(JarHell.java:194)

	at java.nio.file.Files.walkFileTree(Files.java:2670)

	at java.nio.file.Files.walkFileTree(Files.java:2742)

	at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:194)

	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:340)

	... 22 more



java.lang.NullPointerException

	at com.hidden.ola.util.ElasticSearchTest.tearDownAfterClass(ElasticSearchTest.java:95)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.lang.reflect.Method.invoke(Method.java:497)

	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)

	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)

	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)

	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)

	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

So this class com.hidden.ola.agent.restapi.transferobject.ClientRequestPopto is in two places in your classloader:

  • C:\WorkSoft\mvn-repo-security\com\hidden\ola\types\2.5.00-RELEASE\types-2.5.00-RELEASE.jar
  • C:\p4ws\ola\types\target\classes

May be remove the dependency to types-2.5.00-RELEASE?

That's what the log says, however, the same tests were successful when I ran them using maven - not sure if Eclipse is doing something crazy. I can't remove the types dependency: it is required. Is there anyway to disable the jar hell check ?

Hi Anjith,

You have to inspect the eclipse classpath carefully. Maybe you have some other open project and the class is loaded from there. If that is not the case, generate the eclipse artifacts using maven "mvn eclipse:eclipse" and inspect the .classpath.

Cheers,
George

Thanks all. We gave up on this issue. There are many class conflicts from 3rd party jars and it would take a lot of time to resolve all of those. We got the "jar hell" problem only while running unit tests; so we have decided to use external elastic search cluster for tests.

If any one gets into this problem while running unit tests, simply manipulate the java class path such that the class path contains only elasticsearch and lucene jars (don't forget to set the original class path once then es node has been started). This trick worked for me - agree that it's a hack.

String originalClassPath = System.getProperty("java.class.path");
String[] classPathEntries = originalClassPath.split(";");
StringBuilder esClasspath = new StringBuilder();
for (String entry : classPathEntries) {
if (entry.contains("elasticsearch") || entry.contains("lucene")) {
esClasspath.append(entry);
esClasspath.append(";");
}
}
System.setProperty("java.class.path", esClasspath.toString());
node = nodeBuilder().local(true).settings(settings).node();
System.setProperty("java.class.path", originalClassPath);

1 Like