Couldn't resolve error: java.lang.NoSuchFieldError: FAIL_ON_SYMBOL_HASH_OVERFLOW- elasticsearch java client api

[ahlem (trustiser)]

Hello Elastic developer community,

I'm trying to implement autocomple feature using elaticsearch 6.2.4 and its java rest client api on my maven web application which is running under glassfish 4.1
despite all my efforts, I still encounter the same error:
java.lang.NoSuchFieldError: FAIL_ON_SYMBOL_HASH_OVERFLOW

I have read some about it, and made changes accodingly essentielly on project dependencies( inside pom.xml) because all responses that i met, turned arround conflict on jackson library version.
here is the code that I tried to run:

RestHighLevelClient client = new RestHighLevelClient(
        RestClient.builder(
                new HttpHost("localhost", 9200, "http")
                ));
        
         SearchRequest searchRequest = new SearchRequest( "trustiser_suggest" );
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        String[] includeFields = new String[] {"member_mname","member_pic","member_fname","member_lname","topic_label"};
        String[] excludeFields = null;
        searchSourceBuilder.fetchSource(includeFields, excludeFields);
 Map<String, List<? extends ToXContent>> contextMap = new HashMap<>();
    contextMap.put("account_state", Collections.singletonList(CategoryQueryContext.builder().setCategory("active").build()));
        SuggestionBuilder termSuggestionBuilder = SuggestBuilders.completionSuggestion("suggest_member" )
                .prefix( str )
                .contexts(contextMap);
        SuggestBuilder suggestBuilder = new SuggestBuilder();
        suggestBuilder.addSuggestion( "suggest-mem", termSuggestionBuilder );
        
         SuggestionBuilder termSuggestionBuilder1 = SuggestBuilders.completionSuggestion("suggest_topic" )
                .prefix( str )
                .skipDuplicates(true);
        suggestBuilder.addSuggestion( "suggest-top", termSuggestionBuilder1 );
        searchSourceBuilder.suggest( suggestBuilder);

        
        searchRequest.source( searchSourceBuilder );
        SearchResponse searchResponse = null;
        
            searchResponse = client.search( searchRequest );
            
        System.out.println("AutoCompleteMemberTopicccccccccccccccc: "+searchResponse.toString());

my curent pom.xml:

 <dependencies>
        <dependency>
            <groupId>javax</groupId>
            <artifactId>javaee-web-api</artifactId>
            <version>7.0</version>
            <scope>provided</scope>
        </dependency>
 <!-- https://mvnrepository.com/artifact/com.fasterxml.jackson.core/jackson-core -->
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.8.10</version>
    
</dependency>
<!-- https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch -->
<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>6.2.4</version>

</dependency>
<!-- https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-client -->
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-client</artifactId>
    <version>6.2.4</version>
</dependency>


        <dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>6.2.4</version>
</dependency>
<!-- https://mvnrepository.com/artifact/com.googlecode.json-simple/json-simple -->
<dependency>
    <groupId>com.googlecode.json-simple</groupId>
    <artifactId>json-simple</artifactId>
    <version>1.1.1</version>
</dependency>

    </dependencies>

and the error:

java.lang.NoSuchFieldError: FAIL_ON_SYMBOL_HASH_OVERFLOW
	at org.elasticsearch.common.xcontent.json.JsonXContent.<clinit>(JsonXContent.java:57)
	at org.elasticsearch.common.xcontent.XContentFactory.contentBuilder(XContentFactory.java:121)
	at org.elasticsearch.search.suggest.completion.CompletionSuggestionBuilder.contexts(CompletionSuggestionBuilder.java:203)
	at com.trustiser.business.ElasticSearchDao1.AutoCompleteMemberTopic(ElasticSearchDao1.java:50)
	at com.trustiser.service.SearchEngineService.searchMemberCategory(SearchEngineService.java:33)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:151)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:171)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:152)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:104)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:387)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:331)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:103)

Thank you

[ahlem (trustiser)]

and finally the maven tree of the web project

--- maven-dependency-plugin:2.8:tree (default-cli) @ elasticmavenprojet ---
com.trustiser:elasticmavenprojet:war:1.0-SNAPSHOT
+- javax:javaee-web-api:jar:7.0:provided
+- com.fasterxml.jackson.core:jackson-core:jar:2.8.10:compile
+- org.elasticsearch:elasticsearch:jar:6.2.4:compile
|  +- org.elasticsearch:elasticsearch-core:jar:6.2.4:compile
|  +- org.apache.lucene:lucene-core:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-analyzers-common:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-backward-codecs:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-grouping:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-highlighter:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-join:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-memory:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-misc:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-queries:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-queryparser:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-sandbox:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-spatial:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-spatial-extras:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-spatial3d:jar:7.2.1:compile
|  +- org.apache.lucene:lucene-suggest:jar:7.2.1:compile
|  +- org.elasticsearch:securesm:jar:1.2:compile
|  +- org.elasticsearch:elasticsearch-cli:jar:6.2.4:compile
|  |  \- net.sf.jopt-simple:jopt-simple:jar:5.0.2:compile
|  +- com.carrotsearch:hppc:jar:0.7.1:compile
|  +- joda-time:joda-time:jar:2.9.9:compile
|  +- org.yaml:snakeyaml:jar:1.17:compile
|  +- com.fasterxml.jackson.dataformat:jackson-dataformat-smile:jar:2.8.10:compile
|  +- com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar:2.8.10:compile
|  +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.8.10:compile
|  +- com.tdunning:t-digest:jar:3.0:compile
|  +- org.hdrhistogram:HdrHistogram:jar:2.1.9:compile
|  +- org.apache.logging.log4j:log4j-api:jar:2.9.1:compile
|  \- org.elasticsearch:jna:jar:4.5.1:compile
+- org.elasticsearch.client:elasticsearch-rest-client:jar:6.2.4:compile
|  +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile
|  +- org.apache.httpcomponents:httpcore:jar:4.4.5:compile
|  +- org.apache.httpcomponents:httpasyncclient:jar:4.1.2:compile
|  +- org.apache.httpcomponents:httpcore-nio:jar:4.4.5:compile
|  +- commons-codec:commons-codec:jar:1.10:compile
|  \- commons-logging:commons-logging:jar:1.1.3:compile
+- org.elasticsearch.client:elasticsearch-rest-high-level-client:jar:6.2.4:compile
|  +- org.elasticsearch.plugin:parent-join-client:jar:6.2.4:compile
|  |  +- org.locationtech.spatial4j:spatial4j:jar:0.6:compile
|  |  +- com.vividsolutions:jts:jar:1.13:compile
|  |  \- org.apache.logging.log4j:log4j-core:jar:2.9.1:compile
|  +- org.elasticsearch.plugin:aggs-matrix-stats-client:jar:6.2.4:compile
|  \- org.elasticsearch.plugin:rank-eval-client:jar:6.2.4:compile
\- com.googlecode.json-simple:json-simple:jar:1.1.1:compile
   \- junit:junit:jar:4.10:compile
      \- org.hamcrest:hamcrest-core:jar:1.1:compile

[spinscale (Alexander Reelsen)]

is it possible that you are using another version of jackson somewhere? Especially a newer version?

[ahlem (trustiser)]

Hi,
not for this project when I isolate the autocomplete functionality to easiely resolves it.
I shared the dependency tree, and I have checked that 2.8.2 is the only bversion that my project used
also,
the version that I provide (2.8.10) is the one that is used by ES API 6.2.4, because when don't define it explicitly, 2.8.10 is the version that is added!!!

[spinscale (Alexander Reelsen)]

can you check the final artifact instead of the dependency tree regarding your dependencies?

[ahlem (trustiser)]

please, what doyou mean by the final artifact? how can I get it?

[spinscale (Alexander Reelsen)]

at some point you do have a build process that results in a .war or .jar file or something, that you deploy to glassfish - this is the artifact I want you to check.

[ahlem (trustiser)]

great I see, I will check it right now and come back to you with an answer!

[ahlem (trustiser)]

the artifact is a war file, I check the dependency under web-inf/lib, it seems that every thing is ok concerning jackson version.
I attached a screen shots of the projet's dependency

[spinscale (Alexander Reelsen)]

that looks ok.

is it possible that the server where the war is deployed is also using jackson and thus this conflict happens?

[ahlem (trustiser)]

yes, it's the case, the application is deployed on glassfish 4.1 which use jasckson version 2.3

I hope that there is a soft sulution for the problem

[spinscale (Alexander Reelsen)]

you could try to shade the elasticsearch client. See https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-low-usage-shading.html

[ahlem (trustiser)]

thank you for your help, I will take a look to your suggestion and I hope come back with good news

[ahlem (trustiser)]

Hi @spinscale ,
you were right, shading ES hight level client api was the solution of the conflict beteween it and existing jars on glassfish modules ( Jackson-core is one of them).
These are followed steps to implement the solution:

1. Create the shaded dependency of ES:

• Create a java maven project;

• Run clean, build and maven package goal;

• The jar will be created on "elastic shade project location" \target

3. Add the shaded dependency on maven local repository:

• Access to m2 repository from the cmd, the default location is C:\Users"user".m2\repository , and execute the following command to istall the created dependency: mvn install:install-file -Dfile="location of the shaded elastic dependency".jar -DgroupId="shaded dependency groupe id" -DartifactId="shaded dependency acrifact id" -Dversion="version of the shaded dependency" -Dpackaging=jar

4. Add the shaded dependency of elasticsearch on the pom.xml of my maven java web application instead of using elasticsearch hight level rest api directly
5. To avoid log4j error I add a log4j2.properties file on the resources directory of my java web app :

appender.console.type = Console
appender.console.name = console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%m%n

rootLogger.level = info
rootLogger.appenderRef.console.ref = console

could you please take a look to my solution ?

[ahlem (trustiser)]

• The Pom.xml of the java project used to shade ES dependency :

 <modelVersion>4.0.0</modelVersion>
    <groupId>com.trustiser</groupId>
    <artifactId>elasticshade</artifactId>
    <version>6.2.4</version>
    <packaging>jar</packaging>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>6.2.4</version>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.1.0</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>

                            <relocations>
                                <relocation>
                                    <pattern>org.elasticsearch</pattern>
                                    <shadedPattern>hidden.org.elasticsearch</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.apache.lucene</pattern>
                                    <shadedPattern>hidden.org.apache.lucene</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.fasterxml</pattern>
                                    <shadedPattern>hidden.com.fasterxml</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.apache.httpcomponents</pattern>
                                    <shadedPattern>hidden.org.apache.httpcomponents</shadedPattern>
                                </relocation>
                            
                                <relocation>
                                    <pattern>org.hdrhistogram</pattern>
                                    <shadedPattern>hidden.org.hdrhistogram</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.slf4j</pattern>
                                    <shadedPattern>hidden.org.slf4j</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>org.yaml</pattern>
                                    <shadedPattern>hidden.org.yaml</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>net.sf.jopt-simple</pattern>
                                    <shadedPattern>hidden.net.sf.jopt-simple</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>joda-time</pattern>
                                    <shadedPattern>hidden.joda-time</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>commons-logging</pattern>
                                    <shadedPattern>hidden.commons-logging</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>commons-codec</pattern>
                                    <shadedPattern>hidden.commons-codec</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.tdunning</pattern>
                                    <shadedPattern>hidden.com.tdunning</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.github.spullara</pattern>
                                    <shadedPattern>hidden.com.github.spullara</shadedPattern>
                                </relocation>
                                <relocation>
                                    <pattern>com.carrotsearch</pattern>
                                    <shadedPattern>hidden.com.carrotsearch</shadedPattern>
                                </relocation>
                            </relocations>
                            <shadedArtifactAttached>false</shadedArtifactAttached>
                            <artifactSet>
                                <includes>
                                    <include>*:*</include>
                                </includes>
                            </artifactSet>
                            <filters>
                                <filter>
                                    <artifact>*:*</artifact>
                                    <excludes>
                                        <exclude>META-INF/*.SF</exclude>
                                        <exclude>META-INF/*.DSA</exclude>
                                        <exclude>META-INF/*.RSA</exclude>
                                    </excludes>
                                </filter>
                            </filters>                        
                                 
                            <transformers>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.PluginXmlResourceTransformer"/>
                                <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
                                    <mainClass>com.trustiser.elasticshade.App</mainClass>
                                </transformer>
                
                            </transformers>
                        
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    
    </build>