Migrate Elasticsearch Java API to version 5.* in Spring IO Platform application

Elastic Stack Elasticsearch
1

Hi all,

I just want to share my experience migrating a Spring IO app to Elasticsearch version 5.1.1
and maybe get some ideas how to do things in a better way.
The application is using Spring Platform version Athens-SR2 which comes with dependency on org.elasticsearch:elasticsearch version 2.4.3

Luckily, our app does not use spring-data-elasticsearch and work directly with the Elasticsearch API 2.4.3 instead.
Those of you building on top of Spring Data Elasticsearch won't find much useful information here.

So we had our app connecting to 2.4.3 ES cluster but since we plan to migrate the cluster to version 5.* we also had to migrate our Spring app.

There aren't that many changes in the Java API 5.* per se, and most of them are described
here

So I will focus on the dependency issues I got instead. The biggest problem was that our app uses SLF4J+Logback.
First, I had to overwrite the imported ES dependency from Spring Athens-SR2.
Then add the ES Transport Client dependency org.elasticsearch.client:transport
Then, add dependency on Log4j 2 to SLF4J Adapter which routes Log4j2 logging to SLF4J as described
here
The slf4j-api dependency is inherited from the Spring IO Platform so it does not need to be defined explicitly.

These turned out to be the only things I had to do to get up an running with the ES Java API 5.*:

<properties>
    <elasticsearch.version>5.1.1</elasticsearch.version>
    <log4j.version>2.7</log4j.version>
</properties>
<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.spring.platform</groupId>
            <artifactId>platform-bom</artifactId>
            <version>Athens-SR2</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>transport</artifactId>
            <version>${elasticsearch.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-to-slf4j</artifactId>
            <version>${log4j.version}</version>
        </dependency>
....

The biggest problem came when migrating the tests. They are using embedded Elasticsearch instance which has hard dependency on the Log4j 2 logger.
I wanted to avoid relying only on mocks for the ES instance and also overly complicating the integration tests with external ES cluster.
Moreover, the dependency shading which I saw proposed in some places looked like an overkill.
In our tests we create an instance of org.elasticsearch.node.Node and starting it.
The use of ESIntegTestCase from the Elasticsearch Java Testing Framework
brings even more conflicting dependencies with the Spring IO Platform so it was soon discarded.

The workaround I found is to use Log4j 2 instead of SLF4J only during the tests. For that, I used the dependency exclusion feature of the maven-surefire-plugin which we use to run our tests + added dependencies on Log4j 2 with scope test i.e., adding the following to the Maven pom:

  <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>${log4j.version}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-core</artifactId>
        <version>${log4j.version}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.19.1</version>
            <configuration>
                <classpathDependencyExcludes>
                    <classpathDependencyExcludes>org.apache.logging.log4j:log4j-to-slf4j
                    </classpathDependencyExcludes>
                </classpathDependencyExcludes>
            </configuration>
        </plugin>
    </plugins>
</build>

This looks like an ugly hack so any suggestions for something better are very welcome.
I hope this topic can also help others which are looking into using the new Elasticsearch Java API in Spring Boot/Platform applications.

Cheers,
Rumen

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.