Using elasticsearch in tomcat, getting NoClassFoundException

Canappeco · October 18, 2017, 7:54am

Hi, I am using elasticsearch in tomcat. I make a war file and put it in webapps. But now I am getting NoClassFoundException as follow:

esThread is opening ...
18-Oct-2017 15:39:49.537 SEVERE [main] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class [cn.faceall.es.WebContextListener]
 java.lang.NoClassDefFoundError: hidden/org/apache/http/HttpHost
	at cn.faceall.es.threads.ESThread.<init>(ESThread.java:25)
	at cn.faceall.es.WebContextListener.contextInitialized(WebContextListener.java:37)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4643)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5105)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:740)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:716)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:703)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:978)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1850)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:762)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:431)
	at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1578)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:308)
	at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
	at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:424)
	at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:367)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:964)
	at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:839)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1425)
	at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1415)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:134)
	at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:941)
	at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:258)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:422)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:770)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.startup.Catalina.start(Catalina.java:671)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:355)
	at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:495)
Caused by: java.lang.ClassNotFoundException: hidden.org.apache.http.HttpHost
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1275)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1104)
	... 43 more

I am making a thread as follow:

public class WebContextListener implements ServletContextListener {
    @Override
    public void contextInitialized(ServletContextEvent servletContextEvent) {
        try{
            System.out.println("esThread is opening ... ");
            Thread esThread = new Thread(new ESThread());
            esThread.start();
            System.out.println("esThread is on ! ");
            System.out.println("all threads are on ! ");
        } catch (Exception e){
            e.printStackTrace();
        } 
    }
    @Override
    public void contextDestroyed(ServletContextEvent servletContextEvent) {}
}

Can Anyone give me some advice? Thanks!

Canappeco · October 18, 2017, 7:55am

In the thread, I do some simple processing, just saving some documents in ES.

In my pom.xml, I also add <scope>provided</scope>:

    <groupId>cn.faceall</groupId>
    <artifactId>socket-server-rest</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>war</packaging>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.6.1</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-war-plugin</artifactId>
                <version>2.1.1</version>
                <configuration>
                    <failOnMissingWebXml>false</failOnMissingWebXml>
                    <!-- <warName>cargo-tracker</warName> -->
                </configuration>
            </plugin>

            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                    <archive>
                        <manifest>
                            <mainClass></mainClass>
                        </manifest>
                    </archive>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>single</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals><goal>shade</goal></goals>
                        <configuration>
                            <relocations>
                                <relocation>
                                    <pattern>org.apache.http</pattern>
                                    <shadedPattern>hidden.org.apache.http</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.apache.logging</pattern>
                                    <shadedPattern>hidden.org.apache.logging</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.apache.commons.codec</pattern>
                                    <shadedPattern>hidden.org.apache.commons.codec</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.apache.commons.logging</pattern>
                                    <shadedPattern>hidden.org.apache.commons.logging</shadedPattern>
                                </relocation>
                            </relocations>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>org.elasticsearch</groupId>
            <artifactId>elasticsearch</artifactId>
            <version>5.6.3</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>5.6.3</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>5.6.3</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20160810</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-api</artifactId>
            <version>2.8.2</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>log4j-core</artifactId>
            <version>2.8.2</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpasyncclient</artifactId>
            <version>4.1.3</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore</artifactId>
            <version>4.4.6</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
            <version>4.5.3</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpcore-nio -->
        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpcore-nio</artifactId>
            <version>4.4.7</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/commons-logging/commons-logging -->
        <dependency>
            <groupId>commons-logging</groupId>
            <artifactId>commons-logging</artifactId>
            <version>1.2</version>
            <scope>provided</scope>
        </dependency>
        <dependency>
            <groupId>javax.websocket</groupId>
            <artifactId>javax.websocket-api</artifactId>
            <version>1.1</version>
            <scope>provided</scope>
        </dependency>
        <!-- https://mvnrepository.com/artifact/javax.servlet/javax.servlet-api -->
        <dependency>
            <groupId>javax.servlet</groupId>
            <artifactId>javax.servlet-api</artifactId>
            <version>3.1.0</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

Canappeco · October 18, 2017, 7:58am

I am sure I used to run this code with jar -classpath successfully. But once I put it in tomcat, it runs out this Exception.

Thanks!

dadoonet · October 18, 2017, 8:18am

What is this new ESThread() class doing?

I mean that you have here a maven problem I believe. Not specifically an elasticsearch problem.
You are shading libs... Running some part of the code from Tomcat which has I believe its own classloader.
You defined the rest-client as provided. Does it mean that you are adding it manually to the tomcat lib instead of having it inside your war?

BTW I don't think you should depend on elasticsearch-rest-client and elasticsearch.
elasticsearch-rest-high-level-client should be enough.

Canappeco · October 18, 2017, 8:23am

I am sorry I can't directly tell you the content of the ESThread()...

And I try a simple example, getting the same Exception:

public class WebContextListener implements ServletContextListener {

    @Override
    public void contextInitialized(ServletContextEvent servletContextEvent) {
        RestClient restClient = null;
        RestHighLevelClient restHighLevelClient = null;
        try {
            restClient = RestClient.builder(
                    new HttpHost("192.168.1.140", 9200, "http"),
                    new HttpHost("192.168.1.174", 9200, "http"),
                    new HttpHost("192.168.1.231", 9200, "http")
            ).build();
            restHighLevelClient = new RestHighLevelClient(restClient);

            IndexRequest indexRequest = new IndexRequest("posts", "doc", "1").source(jsonBuilder()
                    .startObject()
                    .field("user", "kimchy")
                    .field("postDate", new Date())
                    .field("message", "trying out Elasticsearch")
                    .endObject());

            IndexResponse indexResponse = restHighLevelClient.index(indexRequest);
            if (indexResponse.getResult() == DocWriteResponse.Result.CREATED) {
                System.out.println("the document was created for the first time");
            } else if (indexResponse.getResult() == DocWriteResponse.Result.UPDATED) {
                System.out.println("the document was rewritten as it was already existing");
            }
            ReplicationResponse.ShardInfo shardInfo = indexResponse.getShardInfo();
            if (shardInfo.getTotal() != shardInfo.getSuccessful()) {
                System.out.println("number of successful shards is less than total shards");
            }
            if (shardInfo.getFailed() > 0) {
                for (ReplicationResponse.ShardInfo.Failure failure : shardInfo.getFailures()) {
                    String reason = failure.reason();
                    System.out.println("the reason of the failure is that " + reason);
                }
            }

        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            try {
                restClient.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }
    @Override
    public void contextDestroyed(ServletContextEvent servletContextEvent) {}
}

getting the following Exception:

18-Oct-2017 16:13:42.117 SEVERE [ContainerBackgroundProcessor[StandardEngine[Catalina]]] org.apache.catalina.core.StandardContext.listenerStart Exception sending context initialized event to listener instance of class [cn.faceall.es.WebContextListener]
 java.lang.NoClassDefFoundError: hidden/org/apache/http/HttpHost
	at cn.faceall.es.WebContextListener.contextInitialized(WebContextListener.java:30)
	at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4643)
	at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5105)
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
	at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:740)
	at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:716)
	at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:703)
	at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:978)
	at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1850)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
	at org.apache.catalina.startup.HostConfig.deployWARs(HostConfig.java:762)
	at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:431)
	at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1622)
	at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:304)
	at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
	at org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1172)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1394)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1398)
	at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1366)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: hidden.org.apache.http.HttpHost
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1275)
	at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1104)
	... 23 more

dadoonet · October 18, 2017, 8:26am

You shaded org/apache/http/HttpHost to hidden/org/apache/http/HttpHost. This class does not seem part of your war.

That's all what I can say here.

Canappeco · October 18, 2017, 8:42am

I try to remove <scope>provide</scope>, but it makes no sense...

I am so sorry but can you tell me more about this? Or can you give me just some advice? I really have no idea. Thanks!

dadoonet · October 18, 2017, 10:13am

Why did you shade the libs in the first place?

Canappeco · October 18, 2017, 10:18am

I follow here and add this plugin.

Now I remove this plugin and everything goes correctly. In fact there is no conflict.

Thanks for your help!

dadoonet · October 18, 2017, 10:28am

Yeah. Don't shade if you don't have to solve a conflict problem

system · November 15, 2017, 10:28am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.