Elasticsearch jar hell when writing integration tests

Elastic Stack Elasticsearch
1

Elasticsearch 5 forces me to use the ESIntegTestCase instead of starting my own Node for my integration tests from Java because NodeBuilder is not available anymore. So I try to do things the right way using:

My product uses a large number of dependencies that are also used by Elasticsearch. For example apache.commons codec and logging and Netty.io. Due to the strict nature of ESIntegTestCase it is required to use a clean classpath in the integration tests. There is the jar hell checker to check this.

I understand the purpose of this check and I have read multiple issues and discussions about it. But it makes writing an integration test really impossible.

Example: I depend on netty-all-final4.1.5.jar and elasticsearch depends on the same Netty jars, but not on the final distribution but on the separate jars. To get the integration test working I must either rewrite my code to depend on the separate netty.io jars (netty-buffer, netty-common, netty-codec, etcetera) and use the same approach as elasticsearch uses. Or I have to exclude a number of dependencies manually in my integration test dependencies.
And next time elasticsearch upgrades and uses some other approach I have to redo this error prone work again.

Is there a trick to circumvent the jar hell checker? I'm not writing ES integration tests, I'm writing integration tests for my own product, which has to talk against ES.

java.lang.RuntimeException: found jar hell in test classpath
    at org.elasticsearch.bootstrap.BootstrapForTesting.<clinit>(BootstrapForTesting.java:90)
    at org.elasticsearch.test.ESTestCase.<clinit>(ESTestCase.java:138)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:585)
Caused by: java.lang.IllegalStateException: jar hell!
class: io.netty.util.HashingStrategy$1
jar1: F:\maven\repo\io\netty\netty-common\4.1.5.Final\netty-common-4.1.5.Final.jar
jar2: F:\maven\repo\io\netty\netty-all\4.1.5.Final\netty-all-4.1.5.Final.jar
    at org.elasticsearch.bootstrap.JarHell.checkClass(JarHell.java:291)
    at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:187)
    at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:88)
    at org.elasticsearch.bootstrap.BootstrapForTesting.<clinit>(BootstrapForTesting.java:88)
    ... 4 more

References to jar hell:

1 Like
2

I found some more related information, and perhaps a solution:

should it now be possible in ES 5.0 to disable the check by using -Dtests.jarhell.check=false ?

It seems like the the "tests.jarhell.check" setting never made it to 5.0, only to 2.2 and 2.3 branches.

No if statement around the JarHell check:

    // check for jar hell
    try {
        JarHell.checkJarHell();
    } catch (Exception e) {
        throw new RuntimeException("found jar hell in test classpath", e);
    }

Could anyone confirm this?

3

Hi @escay

We have the same issue as you. We were using the NodeBuilder to run our integration tests against our Search API which is a Spring Boot application. Since the upgrade to Elasticsearch 5 we can no longer use this method.

I'm going to switch to using the ESIntegTestCase instead, but did you find a clean way to resolve the Jar Hell issues?

Thanks,

Dan

4

I tried to solve all dependency problems, but failed due to log4j version 1
versus log4j2 issues. We went back to ES version 2.

The maven plugin to start ES 5 would be my next try. The plugin is
supporting ES 5 in the meantime.

5

See


for the support in the maven plugin. This should allow you to start ES 5
for your integration tests.

6

@escay The Elasticsearch maven plugin works nicely. Thanks for the heads up.