# ElasticSearch 2.0 java api JarHell problems

**URL:** <https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776>\
**Category:** Elasticsearch\
**Created:** [November 17, 2015, 9:42am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776 "2015-11-17T09:42:39Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.elastic.co/u/George)\
**Post date:** [November 17, 2015, 9:42am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/1 "2015-11-17T09:42:39Z")

</div>

Hello everybody,

I'm trying to migrate from elasticsearch 1.6.0 to 2.0.0 using the Java API. The tests were passing with 1.60. However, after I made the tests compile with 2.0.0 my tests are failing when running them locally due to various exceptions generated by JarHell. The question is, can I disable the JarHell check for local testing?

Thanks!

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 17, 2015, 9:54am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/2 "2015-11-17T09:54:39Z")

</div>

If you are running your tests from IntelliJ, that can be the cause.  
Otherwise, I would try to fix any JAR conflict in my project instead of trying to disable JarHell check.

---

<div class="post-metadata">

**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.elastic.co/u/George)\
**Post date:** [November 17, 2015, 10:28am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/3 "2015-11-17T10:28:11Z")

</div>

Hello,

Thanks for the response.  
Actually , I'm running the tests using gradle 2.6, jdk 7 also tried it with 8.  
Could you please tell me how could I work around such an issue ?

java.lang.RuntimeException: found jar hell in test classpath  
Caused by:  
java.lang.IllegalStateException: jar hell!  
class: org.apache.hadoop.yarn.factories.package-info  
jar1: /home/george/.gradle/caches/modules-2/files-2.1/org.apache.hadoop/hadoop-yarn-api/2.5.0-cdh5.3.0/7542d2b9b89956291d8fb4294aab143adf4f5252/hadoop-yarn-api-2.5.0-cdh5.3.0.jar  
jar2: /home/george/.gradle/caches/modules-2/files-2.1/org.apache.hadoop/hadoop-yarn-common/2.5.0-cdh5.3.0/ab28ad57af245463f66a5da314dffc684fe9cc13/hadoop-yarn-common-2.5.0-cdh5.3.0.jar

This is about the package-info that happens to live in the same package in two different jars.

Thanks!

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [November 17, 2015, 11:00am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/4 "2015-11-17T11:00:10Z")

</div>

The "identical jar" is not a jar hell by Oracle's definition for a class path, but it was refused to fix: [https://github.com/elastic/elasticsearch/issues/13604](https://github.com/elastic/elasticsearch/issues/13604)

So you have to do some very weird acrobatics in a gradle build, something like this for integration testing

```auto
task integrationTest(type: Test, dependsOn: ['unpackPlugin']) {
    testClassesDir = sourceSets.integrationTest.output.classesDir
    // the test dependencies including Elasticsearch jars
    classpath = configurations.integrationTestCompile
    // the dependencies of this plugin as jars from the plugin zip
    classpath += fileTree('plugins/<name>').include('*.jar')
    // our plugin classes and resources
    classpath += sourceSets.integrationTest.output
    // without removing identical jars from classpath, Elasticsearch whines about a "jar hell"
    classpath -= configurations.releaseJars
    outputs.upToDateWhen { false }
    systemProperty 'path.home', projectDir.absolutePath
}

```

The complete gradle build file is at [https://github.com/jprante/elasticsearch-webapp/blob/master/build.gradle](https://github.com/jprante/elasticsearch-webapp/blob/master/build.gradle)  
It's a groovy plugin, but the 'identical jar' challenge also applies to Java plugin builds with gradle.

---

<div class="post-metadata">

**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.elastic.co/u/George)\
**Post date:** [November 17, 2015, 11:27am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/5 "2015-11-17T11:27:36Z")

</div>

Hello,

But to be honest this JarHell is really restrictive. I mean take for example this package-info, which is actually not used at runtime. Why would you want to fail the build because of that? I should be able to decide as a client if this is fine with me or not. Elasticsearch should not decide about incompatibilities in other libraries in the same classpath, in my opinion.

@jprante Can you please suggest some way to run the tests locally and avoid JarHell? I have a simple test with no plugins, no nothing, only extending the ESIntegTestCase creating a mapping and retrievering it. I'm using the hadoop dependecy because I have a custom definition of the fields. Because I see that your gradle integrationTest depends on some unpackPlugin, which is not an option for me.

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [November 17, 2015, 11:49am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/6 "2015-11-17T11:49:55Z")

</div>

@George I never tried `ESIntegTestCase`, it looks too tricky to me

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [November 17, 2015, 12:36pm UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/7 "2015-11-17T12:36:04Z")

</div>

I see. I tried also once to play with Hadoop but it's really a JAR hell there!

Not sure about what you can do. I think @costin solved that issue on his end for the hadoop plugin. @costin how did you fix that?

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [November 17, 2015, 12:52pm UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/8 "2015-11-17T12:52:47Z")

</div>

Load Hadoop into its own `ClassLoader` so that it's not part of the ES/JarHell classpath.

---

<div class="post-metadata">

**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.elastic.co/u/George)\
**Post date:** [November 17, 2015, 1:09pm UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/9 "2015-11-17T13:09:31Z")

</div>

Hi Costin,

Thanks for the response.  
Can I do that in gradle when I run the test? Do you have some snippet or some example somewhere?

@dadoonet thanks for asking costin.  
Thanks!

---

<div class="post-metadata">

**Author:** ![costin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/costin/32/44950_2.png) [@costin](https://discuss.elastic.co/u/costin)\
**Post date:** [November 17, 2015, 1:37pm UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/10 "2015-11-17T13:37:01Z")

</div>

Not that I'm aware of - it's doable probably through some type of plugin however note that the JarHell can occur easily at runtime as well.  
You can see the `ClassLoader` solution (which is Java Security Manager friendly as it does not require extra permissions) is here, in the [repository-hdfs repo](https://github.com/elastic/elasticsearch-hadoop/blob/a78cd8c5621e9939de3895e89a869aa99be37553/repository-hdfs/src/main/java/org/elasticsearch/plugin/hadoop/hdfs/HdfsPlugin.java)

---

<div class="post-metadata">

**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\
**Post date:** [January 15, 2016, 5:47am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/11 "2016-01-15T05:47:48Z")

</div>

I'm getting the same problem while running tests from Eclipse - running from maven works. Is there any solution to fix this issue in Eclipse ?

Caused by: java.lang.IllegalStateException: jar hell!  
class: com.**_..agent.restapi.transferobject.ClientRequestPopto  
jar1: C:\WorkSoft\mvn-repo-security\com\*\*_ **\2.5.00-RELEASE\*** _-2.5.00-RELEASE.jar  
jar2: C:\p4ws\ola\*_**\*\*\target\classes  
at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:267)

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 15, 2016, 6:28am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/12 "2016-01-15T06:28:10Z")

</div>

Fix your classpath. You can't have the same class twice in the classloader.

---

<div class="post-metadata">

**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\
**Post date:** [January 15, 2016, 6:39am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/13 "2016-01-15T06:39:03Z")

</div>

@dadoonet, I have checked the classpath - generate by m2e plugin- and it doesn't have any duplicate entries - classpath is generated my maven eclipse integration. And, I'm gettng this problem **only** while running tests from the Eclipse. Any clues on what is going wrong ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 15, 2016, 7:20am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/14 "2016-01-15T07:20:02Z")

</div>

Can you print the full log?

---

<div class="post-metadata">

**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\
**Post date:** [January 15, 2016, 7:39am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/15 "2016-01-15T07:39:14Z")

</div>

Here is the trace:

```
TestBWElasticSearchSink.testProcess_BulkRequestWithChunks1
Unrooted Tests
com.hidden.ola.collection.payload.TestBWElasticSearchSink
java.lang.IllegalStateException: failed to load bundle [] due to jar hell

	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:342)

	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:113)

	at org.elasticsearch.node.Node.<init>(Node.java:144)

	at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)

	at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:152)

	at com.hidden.ola.util.ElasticSearchTest.setUpBeforeClass(ElasticSearchTest.java:79)

	at com.hidden.ola.collection.payload.TestBWElasticSearchSink.setUpBeforeClass(TestBWElasticSearchSink.java:46)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.lang.reflect.Method.invoke(Method.java:497)

	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)

	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)

	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)

	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)

	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)

	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

Caused by: java.lang.IllegalStateException: jar hell!

class: com.hidden.ola.agent.restapi.transferobject.ClientRequestPopto

jar1: C:\WorkSoft\mvn-repo-security\com\hidden\ola\types\2.5.00-RELEASE\types-2.5.00-RELEASE.jar

jar2: C:\p4ws\ola\types\target\classes

	at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:267)

	at org.elasticsearch.bootstrap.JarHell$1.visitFile(JarHell.java:201)

	at org.elasticsearch.bootstrap.JarHell$1.visitFile(JarHell.java:194)

	at java.nio.file.Files.walkFileTree(Files.java:2670)

	at java.nio.file.Files.walkFileTree(Files.java:2742)

	at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:194)

	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:340)

	... 22 more

java.lang.NullPointerException

	at com.hidden.ola.util.ElasticSearchTest.tearDownAfterClass(ElasticSearchTest.java:95)

	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

	at java.lang.reflect.Method.invoke(Method.java:497)

	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)

	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)

	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)

	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)

	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)

	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)

	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 15, 2016, 7:59am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/16 "2016-01-15T07:59:42Z")

</div>

So this class `com.hidden.ola.agent.restapi.transferobject.ClientRequestPopto` is in two places in your classloader:

- `C:\WorkSoft\mvn-repo-security\com\hidden\ola\types\2.5.00-RELEASE\types-2.5.00-RELEASE.jar`
- `C:\p4ws\ola\types\target\classes`

May be remove the dependency to `types-2.5.00-RELEASE`?

---

<div class="post-metadata">

**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\
**Post date:** [January 15, 2016, 8:32am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/17 "2016-01-15T08:32:11Z")

</div>

That's what the log says, however, the same tests were successful when I ran them using maven - not sure if Eclipse is doing something crazy. I can't remove the types dependency: it is required. Is there anyway to disable the jar hell check ?

---

<div class="post-metadata">

**Author:** ![George](https://avatars.discourse-cdn.com/v4/letter/g/71e660/32.png) [@George](https://discuss.elastic.co/u/George)\
**Post date:** [January 15, 2016, 8:50am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/18 "2016-01-15T08:50:50Z")

</div>

Hi Anjith,

You have to inspect the eclipse classpath carefully. Maybe you have some other open project and the class is loaded from there. If that is not the case, generate the eclipse artifacts using maven "mvn eclipse:eclipse" and inspect the .classpath.

Cheers,  
George

---

<div class="post-metadata">

**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\
**Post date:** [January 15, 2016, 11:04am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/19 "2016-01-15T11:04:37Z")

</div>

Thanks all. We gave up on this issue. There are many class conflicts from 3rd party jars and it would take a lot of time to resolve all of those. We got the "jar hell" problem only while running unit tests; so we have decided to use external elastic search cluster for tests.

---

<div class="post-metadata">

**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\
**Post date:** [January 17, 2016, 5:24am UTC](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776/20 "2016-01-17T05:24:50Z")

</div>

If any one gets into this problem while running unit tests, simply manipulate the java class path such that the class path contains only elasticsearch and lucene jars (don't forget to set the original class path once then es node has been started). This trick worked for me - agree that it's a hack.

> String originalClassPath = System.getProperty("java.class.path");  
> String classPathEntries = originalClassPath.split(";");  
> StringBuilder esClasspath = new StringBuilder();  
> for (String entry : classPathEntries) {  
> if (entry.contains("elasticsearch") || entry.contains("lucene")) {  
> esClasspath.append(entry);  
> esClasspath.append(";");  
> }  
> }  
> System.setProperty("java.class.path", esClasspath.toString());  
> node = nodeBuilder().local(true).settings(settings).node();  
> System.setProperty("java.class.path", originalClassPath);

[Next page](https://discuss.elastic.co/t/elasticsearch-2-0-java-api-jarhell-problems/34776.md?page=2)