Logging classpath issues with ES 5.x

Elastic Stack Elasticsearch
1

Hi all,

I am trying to use ES 5.1.1 in a dead simple application and hitting several walls along the way, especially regarding the logging dependencies.

I am using SLF4J with Logback for my own code, but ES expects to find Log4j instead - fine, I can use the log4j-to-slf4j bridge :slight_smile: - or so I thought.

My production code works fine with the following POM dependencies:

    <dependency>
        <groupId>org.elasticsearch.client</groupId>
        <artifactId>x-pack-transport</artifactId>
        <version>5.1.1</version>
    </dependency>
    <dependency>
        <groupId>ch.qos.logback</groupId>
        <artifactId>logback-classic</artifactId>
        <version>1.2.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-api</artifactId>
        <version>2.7</version>
    </dependency>
    <dependency>
        <groupId>org.apache.logging.log4j</groupId>
        <artifactId>log4j-to-slf4j</artifactId>
        <version>2.7</version>
    </dependency>

However, when I try to create a local node in my integration test code, I get the following stack trace:

Exception in thread "elasticsearch[tmp-es--274345119][clusterService#updateTask][T#1]" java.lang.NoClassDefFoundError: org/apache/logging/log4j/core/config/Configurator
    at org.elasticsearch.common.logging.Loggers.setLevel(Loggers.java:149)
    at org.elasticsearch.common.logging.Loggers.setLevel(Loggers.java:144)
    at org.elasticsearch.index.SearchSlowLog.setLevel(SearchSlowLog.java:114)
    at org.elasticsearch.index.SearchSlowLog.<init>(SearchSlowLog.java:109)
    at org.elasticsearch.index.IndexModule.<init>(IndexModule.java:127)
    at org.elasticsearch.indices.IndicesService.createIndexService(IndicesService.java:421)
    at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:394)
    at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:352)
    at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
    at org.elasticsearch.cluster.service.ClusterService.runTasksForExecutor(ClusterService.java:581)
    at org.elasticsearch.cluster.service.ClusterService$UpdateTask.run(ClusterService.java:920)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:238)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:201)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.core.config.Configurator
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 17 more

I have tried to add log4j-core as a test-only dependency, but then ES complains again:

java.lang.ClassCastException: org.apache.logging.slf4j.SLF4JLoggerContext cannot be cast to org.apache.logging.log4j.core.LoggerContext

    at org.apache.logging.log4j.core.LoggerContext.getContext(LoggerContext.java:187)
    at org.apache.logging.log4j.core.config.Configurator.setLevel(Configurator.java:291)
    at org.elasticsearch.common.logging.Loggers.setLevel(Loggers.java:149)
    at org.elasticsearch.common.logging.Loggers.setLevel(Loggers.java:144)
    at org.elasticsearch.index.SearchSlowLog.setLevel(SearchSlowLog.java:114)
    at org.elasticsearch.index.SearchSlowLog.<init>(SearchSlowLog.java:109)
    at org.elasticsearch.index.IndexModule.<init>(IndexModule.java:127)
    at org.elasticsearch.indices.IndicesService.createIndexService(IndicesService.java:421)
    at org.elasticsearch.indices.IndicesService.createIndex(IndicesService.java:394)
    at org.elasticsearch.cluster.metadata.MetaDataCreateIndexService$1.execute(MetaDataCreateIndexService.java:352)
    at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
    at org.elasticsearch.cluster.service.ClusterService.runTasksForExecutor(ClusterService.java:581)
    at org.elasticsearch.cluster.service.ClusterService$UpdateTask.run(ClusterService.java:920)
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:458)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:238)
    at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:201)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)

Is there a way to get out of this dependency hell, without me having to change my logging framework of choice?

Thanks!

2

Does the documentation here not work for you? https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/_using_another_logger.html

3

Hello David,

Thanks for your answer.

It works correctly in the production code - I can start my service and see the logs I want - but it fails when trying to start a local node in my (JUnit-based) integration test.

FYI this is how I create my local node:

    String nodeName = "tmp-es-" + new Random(System.currentTimeMillis()).nextInt();
    Path tmpDir = Files.createTempDirectory("tmp-es");
    tmpDir.toFile().deleteOnExit();

    org.elasticsearch.common.settings.Settings settings = org.elasticsearch.common.settings.Settings.builder()
            .put("cluster.name", nodeName)
            .put("node.name", nodeName)
            // limit the number of threads created (see org.elasticsearch.common.util.concurrent.EsExecutors)
            .put("processors", 1)
            .put("http.enabled", false)
            .put("transport.type", "local")
            .put("path.home", tmpDir)
            .build();

    node = new Node(settings).start();
    checkState(!node.isClosed());

    // wait for node to be ready
    ClusterHealthResponse clusterHealthResponse =
            node.client().admin().cluster().prepareHealth().setWaitForGreenStatus().execute().actionGet();
    assertThat(clusterHealthResponse.getStatus()).isEqualTo(ClusterHealthStatus.GREEN);

    // delete the indices (should not exist)
    DeleteIndexResponse response = node.client().admin().indices().prepareDelete("_all").get();
    checkState(response.isAcknowledged());

Thanks!

4

Starting nodes from the jvm is not supported.
You should prefer real integration tests.

Have a look at https://github.com/alexcojocaru/elasticsearch-maven-plugin

I just started using it today and it's really great.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.