Class Not Found, for a local jar I try to add to a plugin

Junaid03 · October 18, 2017, 10:37am

I am getting a class not found exception for a local jar file I add to my project using Gradle.

So here are more details:

Using mvn command to install the jar to local repo:

mvn install:install-file -Dfile=/Users/prime/Downloads/AdAuthenticationService.jar -DgroupId=com.xxx.service -DartifactId=AdAuthenticationService -Dversion=1.0 -Dpackaging=jar

I can now verify the file is in my .m2 repo:

In my build gradle:

repositories {
    jcenter()
    mavenLocal()
    maven {
        url "https://artifacts.elastic.co/maven"
    }
}

dependencies {
    compile ("com.xxx.service:AdAuthenticationService:1.0")
 }

Now I list all my Gradle dependencies (Runtime) using

gradle dependencies --configuration runtime

And below is the result:

=======================================
Elasticsearch Build Hamster says Hello!
=======================================
  Gradle Version        : 3.3
  OS Info               : Mac OS X 10.12.5 (x86_64)
  JDK Version           : Oracle Corporation 1.8.0_111 [Java HotSpot(TM) 64-Bit Server VM 25.111-b14]
  JAVA_HOME             : /Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home
Incremental java compilation is an incubating feature.
:dependencies

------------------------------------------------------------
Root project
------------------------------------------------------------

runtime - Runtime dependencies for source set 'main'.
+--- com.xxx.service:AdAuthenticationService:1.0
+--- org.elasticsearch:elasticsearch:5.5.1
|    +--- org.apache.lucene:lucene-core:6.6.0
|    +--- org.apache.lucene:lucene-analyzers-common:6.6.0
|    +--- org.apache.lucene:lucene-backward-codecs:6.6.0
|    +--- org.apache.lucene:lucene-grouping:6.6.0
|    +--- org.apache.lucene:lucene-highlighter:6.6.0
|    +--- org.apache.lucene:lucene-join:6.6.0
|    +--- org.apache.lucene:lucene-memory:6.6.0
|    +--- org.apache.lucene:lucene-misc:6.6.0
|    +--- org.apache.lucene:lucene-queries:6.6.0
|    +--- org.apache.lucene:lucene-queryparser:6.6.0
|    +--- org.apache.lucene:lucene-sandbox:6.6.0
|    +--- org.apache.lucene:lucene-spatial:6.6.0
|    +--- org.apache.lucene:lucene-spatial-extras:6.6.0
|    +--- org.apache.lucene:lucene-spatial3d:6.6.0
|    +--- org.apache.lucene:lucene-suggest:6.6.0
|    +--- org.elasticsearch:securesm:1.1
|    +--- net.sf.jopt-simple:jopt-simple:5.0.2
|    +--- com.carrotsearch:hppc:0.7.1
|    +--- joda-time:joda-time:2.9.5
|    +--- org.yaml:snakeyaml:1.15
|    +--- com.fasterxml.jackson.core:jackson-core:2.8.6
|    +--- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:2.8.6
|    +--- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.8.6
|    +--- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:2.8.6
|    +--- com.tdunning:t-digest:3.0
|    +--- org.hdrhistogram:HdrHistogram:2.1.9
|    +--- org.apache.logging.log4j:log4j-api:2.8.2
|    \--- org.elasticsearch:jna:4.4.0
+--- org.elasticsearch.plugin:x-pack-api:5.5.1
|    +--- org.elasticsearch.plugin:transport-netty3-client:5.5.1
|    |    \--- io.netty:netty:3.10.6.Final
|    +--- org.elasticsearch.plugin:transport-netty4-client:5.5.1
|    |    +--- io.netty:netty-buffer:4.1.11.Final
|    |    +--- io.netty:netty-codec:4.1.11.Final
|    |    +--- io.netty:netty-codec-http:4.1.11.Final
|    |    +--- io.netty:netty-common:4.1.11.Final
|    |    +--- io.netty:netty-handler:4.1.11.Final
|    |    +--- io.netty:netty-resolver:4.1.11.Final
|    |    \--- io.netty:netty-transport:4.1.11.Final
|    +--- com.unboundid:unboundid-ldapsdk:3.2.0
|    +--- org.bouncycastle:bcprov-jdk15on:1.55
|    +--- org.bouncycastle:bcpkix-jdk15on:1.55
|    +--- com.googlecode.owasp-java-html-sanitizer:owasp-java-html-sanitizer:r239
|    +--- com.google.guava:guava:16.0.1
|    +--- com.sun.mail:javax.mail:1.5.3
|    +--- javax.activation:activation:1.1
|    +--- org.elasticsearch.client:rest:5.5.1
|    |    +--- org.apache.httpcomponents:httpclient:4.5.2
|    |    +--- org.apache.httpcomponents:httpcore:4.4.5
|    |    +--- org.apache.httpcomponents:httpasyncclient:4.1.2
|    |    +--- org.apache.httpcomponents:httpcore-nio:4.4.5
|    |    +--- commons-codec:commons-codec:1.10
|    |    \--- commons-logging:commons-logging:1.1.3
|    +--- org.elasticsearch.client:sniffer:5.5.1
|    |    +--- org.elasticsearch.client:rest:5.5.1 (*)
|    |    +--- org.apache.httpcomponents:httpclient:4.5.2
|    |    +--- org.apache.httpcomponents:httpcore:4.4.5
|    |    +--- commons-codec:commons-codec:1.10
|    |    +--- commons-logging:commons-logging:1.1.3
|    |    \--- com.fasterxml.jackson.core:jackson-core:2.8.6
|    \--- net.sf.supercsv:super-csv:2.4.0
\--- org.elasticsearch:jna:4.4.0

(*) - dependencies omitted (listed previously)

BUILD SUCCESSFUL

I can see the dependency in question listed at top of the above list. However when I deploy and run the program I get a class not found exception:

Caused by: java.lang.ClassNotFoundException: com.xxx.service.delegate.ADAuthenticationDelegate

I have the x-pack custom security module as my base code, which is on elastic github page!

Not sure what do I miss here?

Also in elasticsearch documentation I read:

Note that only jar files in the elasticsearch directory are added to the classpath for the plugin! If you need other resources, package them into a resources jar.

However the meaning of resources jar is not completely clear to me!

dadoonet · October 18, 2017, 12:09pm

Moving the thread to #x-pack although I'm unsure if it's a gradle concern or an x-pack one.

Junaid03 · October 18, 2017, 12:28pm

I manually pushed the jar file to the x-pack dir under my elastic installation and the issue is resolved!

Vivek_Unni · November 10, 2017, 1:24pm

My build is still failing even after moving the jar manually to x-pack directory under elasticsearch

[2017-11-10T08:13:54,043][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] fatal error in thread [main], exiting
| java.lang.NoClassDefFoundError: org/elasticsearch/example/realm/JavaInMemoryCache
| at org.elasticsearch.example.realm.CustomRealm.(CustomRealm.java:92) ~[?:?]
| at org.elasticsearch.example.realm.CustomRealmFactory.create(CustomRealmFactory.java:38) ~[?:?]
| at org.elasticsearch.example.realm.CustomRealmFactory.create(CustomRealmFactory.java:29) ~[?:?]
| at org.elasticsearch.xpack.security.authc.Realms.initRealms(Realms.java:179) ~[?:?]
| at org.elasticsearch.xpack.security.authc.Realms.(Realms.java:69) ~[?:?]
| at org.elasticsearch.xpack.security.Security.createComponents(Security.java:382) ~[?:?]
| at org.elasticsearch.xpack.XPackPlugin.createComponents(XPackPlugin.java:276) ~[?:?]
| at org.elasticsearch.node.Node.lambda$new$7(Node.java:412) ~[elasticsearch-5.6.0.jar:5.6.0]

Task :integTestCluster#stop
kill 1937: No such process

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.

Junaid03 · November 10, 2017, 2:14pm

Did you refresh your Gradle? If you are using IntelliJ you must see the jar under the external Libs dir. Also make sure that you have the jar in your /elasticsearch/plugin/x-pack -- if you are testing from an IDE. Otherwise you need to prepare this environment at deployment node.

Vivek_Unni · November 10, 2017, 2:15pm

I just rerun the gradle. Is there anything else I need to do?

Junaid03 · November 10, 2017, 2:17pm

Are you testing this in the IDE or are you deploying the plugin in actual environment?

Vivek_Unni · November 10, 2017, 2:19pm

I'm building it in the dev environment and not in IDE

Junaid03 · November 10, 2017, 2:21pm

Something seems to be wrong with the path org/elasticsearch/example/realm/JavaInMemoryCache. org.elasticsearch.example is the provided package. How are you adding your class here?

Vivek_Unni · November 10, 2017, 2:24pm

package is org.elasticsearch.example.realm and Class is JavaInMemoryCache<K, T>
I ran this mvn command to dev environment's local repo

./mvn install:install-file -Dfile=ama-user-token.jar -DgroupId=org.elasticsearch.example.realm -DartifactId=ama-user-token -Dversion=0.0.1 -Dpackaging=jar

Any issues in it?

Junaid03 · November 10, 2017, 2:28pm

The .m2 repo did not help me much. That is why I had to put this under the x-pack folder. However I had to undergo a range of things for the jar to be proper gradle jar. Like adding the sha and other txt licenses. Not sure if that makes a difference. I suggest you build the setup in an IDE and then move the jar to x-pack folder. Cant say much without access to the environment. having said that "org.elasticsearch.example.realm" is a container package of the example. Your jar might have different package name? Or are you just adding a class?

Vivek_Unni · November 10, 2017, 2:31pm

I m accessing the class using this jar and jar also has the same package name as in group id.
Below is starting snippet of the compiled class that I m trying to access from the jar.

package org.elasticsearch.example.realm;

import java.util.ArrayList;
import org.apache.commons.collections.MapIterator;
import org.apache.commons.collections.map.LRUMap;

public class JavaInMemoryCache<K, T>
{
private long timeToLive;
private LRUMap cacheMap;

protected class CacheObject

TimV · November 12, 2017, 11:53pm

It sounds like you are attacking this from the wrong angle.

The process for an X-Pack extension is as follows:

You write the extension
You package your code and all necessary jar files into an extension bundle
You install that extension bundle using bin/x-pack/extension install ...
You update your elasticsearch.yml config to make use of your extension
You restart Elasticsearch.

If you've followed that sequence of steps and are getting NoClassDefFoundError errors at step 5, then the problem is back at step 2 - you haven't bundled everything up correctly. Don't go messing around with manually copying jar files into ES directories, that's just asking for trouble. Go back to step 2 and work out what's wrong with your extension bundle.

What does your extension jar/zip file look like?
Run unzip -l on it, and post the contents for us.

Vivek_Unni · November 13, 2017, 8:55am

Hi Tim,

Under /elasticsearch-5.6.0/plugins/x-pack/extensions/customrealm, Below are the contents in the jar file

Also, would like know more about the 2nd pointer as how do I package my external jar to an extension bundle. Any specific things to be done in the Gradle apart from setting dependencies as below.

dependencies {
//some predefined dependencies
provided "org.elasticsearch.example.realm:ama-user-token:0.0.1"

testCompile "org.elasticsearch.example.realm:ama-user-token:0.0.1"

}

-bash-4.1$ $JAVA_HOME/bin/jar -xvf x-pack-custom-realm-extension-example-5.6.0.jar
created: META-INF/
inflated: META-INF/MANIFEST.MF
created: org/
created: org/elasticsearch/
created: org/elasticsearch/example/
inflated: org/elasticsearch/example/CustomRealmExtension.class
created: org/elasticsearch/example/realm/
inflated: org/elasticsearch/example/realm/CustomAuthenticationFailureHandler.class
inflated: org/elasticsearch/example/realm/CustomCachingRealm$UserHolder.class
inflated: org/elasticsearch/example/realm/CustomCachingRealm.class
inflated: org/elasticsearch/example/realm/CustomRealm$InfoHolder.class
inflated: org/elasticsearch/example/realm/CustomRealm$AMAStandardMemoryCache$CacheObject.class
inflated: org/elasticsearch/example/realm/CustomRealm$AMAStandardMemoryCache$1.class
inflated: org/elasticsearch/example/realm/CustomRealm$AMAStandardMemoryCache.class
inflated: org/elasticsearch/example/realm/CustomRealm.class
inflated: org/elasticsearch/example/realm/CustomCachingRealmFactory.class
inflated: org/elasticsearch/example/realm/CustomRealmFactory.class
inflated: x-pack-extension-descriptor.properties
inflated: META-INF/LICENSE.txt
inflated: META-INF/NOTICE.txt

TimV · November 13, 2017, 10:56pm

You should build your extension using whatever tooling you are most comfortable with.
Our sample uses gradle because it builds on top of the same build infrastructure we use in the rest of the Elasticsearch products, but unless you know gradle well, or are willing to invest some time into learning it, you'd probably be better using your own tooling (whether that be maven, gradle, ant, or something else). The sample is intended as a sample, not as a base template for your development.

If you have a complex realm with its own dependencies, then you should package it as a zip file.

Inside that zip file you include all the jars that you need that are not already part of Elasticsearch or X-Pack, and your x-pack-extension-descriptor.properties

In the sample you can run gradle buildZip that will build such a zip file - but it doesn't include any dependencies because the same realm doesn't rely on any additional jar files.

system · December 11, 2017, 10:56pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Action class not found on jar file, but works on Eclipse, how is that? Elasticsearch	3	1566	March 3, 2020
Class file for org.elasticsearch.common.xcontent.ToXContentObject not found Elasticsearch	5	8346	May 6, 2020
Can't get started with the ElasticSearch Java API classes Elasticsearch language-clients	3	841	January 12, 2023
Error - XPack Class not found Exception in java Elasticsearch	14	7301	January 11, 2017
ClassNotFoundException: org.elasticsearch.plugins.ActionPlugin Elasticsearch	4	4224	February 4, 2017

Class Not Found, for a local jar I try to add to a plugin

Related Topics