When I am using 6.x REST CLient with 7.X ES, when accessing 7.x ES,I got java.lang.ClassNotFoundException: org.elasticsearch.search.aggregations.metrics.cardinality.ParsedCardinality since I have a Cardinality Aggregation.
When I am using both 6.x REST Client and ES lib, when access 7.x ES, some of the aggregation result on 7.X ES is not correct.
Here is the stack trace:
ElasticsearchStatusException[Elasticsearch exception [type=index_not_found_exception, reason=no such index [xxx-api]]]
at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:2053)
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:2030)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1777)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1734)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1696)
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:1092)
...
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [xxx], URI [/xxx-api/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512], status line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [xxx-api]","resource.type":"index_or_alias","resource.id":"xxx-api","index_uuid":"_na_","index":"xxx-api"}],"type":"index_not_found_exception","reason":"no such index [xxx-api]","resource.type":"index_or_alias","resource.id":"xxx-api","index_uuid":"_na_","index":"xxx-api"},"status":404}
at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:936)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:233)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1764)
... 113 more
Caused by: org.elasticsearch.client.ResponseException: method [POST], host [xxx], URI [/xxx-api/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&search_type=query_then_fetch&batched_reduce_size=512], status line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [xxx-api]","resource.type":"index_or_alias","resource.id":"xxx-api","index_uuid":"_na_","index":"xxx-api"}],"type":"index_not_found_exception","reason":"no such index [xxx-api]","resource.type":"index_or_alias","resource.id":"xxx-api","index_uuid":"_na_","index":"xxx-api"},"status":404}
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:552)
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:537)
at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:123)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:121)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
... 1 more
2019-08-13 16:36:13.750 DEBUG 4688 --- [ XNIO-2 task-1] xxx.errors.ExceptionTranslator : An unexpected error occurred: null
java.lang.NullPointerException: null
at xxx.dao.impl.QueryService.search(QueryService.java:237)
at xxx.dao.impl.QueryService.search(QueryService.java:100)
at xxx.web.rest.SearchResource.getSummary(SearchResource.java:75)
at xxx.web.rest.SearchResource$$FastClassBySpringCGLIB$$cec506a1.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
...
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-08-13 16:36:13.788 WARN 4688 --- [ XNIO-2 task-1] .m.m.a.ExceptionHandlerExceptionResolver : Resolved exception caused by Handler execution: java.lang.NullPointerException
There is no easy way to reproduce the error. But the code is like following: ExpirationDateUtil.getAggregationBuilders(entry.getValue().getPath()).forEach(edrSubAggBuilder -> {
In the stacktrace that you posted, I see an index not found exception being returned, and a null pointer exception in your client code. I don't see any class not found.
Sorry, my bad. Did not pay attention to the error. Here is the correct trace:
org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/search/aggregations/metrics/cardinality/ParsedCardinality
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:982)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
...
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NoClassDefFoundError: org/elasticsearch/search/aggregations/metrics/cardinality/ParsedCardinality
at org.elasticsearch.client.RestHighLevelClient.lambda$getDefaultNamedXContents$19(RestHighLevelClient.java:2087)
at org.elasticsearch.common.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:141)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.namedObject(AbstractXContentParser.java:415)
at org.elasticsearch.common.xcontent.XContentParserUtils.parseTypedKeysObject(XContentParserUtils.java:153)
at org.elasticsearch.search.aggregations.bucket.range.ParsedRange$ParsedBucket.parseRangeBucketXContent(ParsedRange.java:182)
at org.elasticsearch.search.aggregations.bucket.range.ParsedDateRange$ParsedBucket.fromXContent(ParsedDateRange.java:71)
at org.elasticsearch.search.aggregations.bucket.range.ParsedDateRange.lambda$static$0(ParsedDateRange.java:41)
at org.elasticsearch.search.aggregations.ParsedMultiBucketAggregation.lambda$declareMultiBucketAggregationFields$0(ParsedMultiBucketAggregation.java:75)
at org.elasticsearch.common.xcontent.ObjectParser.parseValue(ObjectParser.java:314)
at org.elasticsearch.common.xcontent.ObjectParser.parseArray(ObjectParser.java:308)
at org.elasticsearch.common.xcontent.ObjectParser.parseSub(ObjectParser.java:340)
at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:168)
at org.elasticsearch.common.xcontent.ObjectParser.parse(ObjectParser.java:131)
at org.elasticsearch.search.aggregations.bucket.range.ParsedDateRange.fromXContent(ParsedDateRange.java:46)
at org.elasticsearch.client.RestHighLevelClient.lambda$getDefaultNamedXContents$55(RestHighLevelClient.java:2124)
at org.elasticsearch.common.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:141)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.namedObject(AbstractXContentParser.java:415)
at org.elasticsearch.common.xcontent.XContentParserUtils.parseTypedKeysObject(XContentParserUtils.java:153)
at org.elasticsearch.search.aggregations.bucket.ParsedSingleBucketAggregation.parseXContent(ParsedSingleBucketAggregation.java:86)
at org.elasticsearch.search.aggregations.bucket.filter.ParsedFilter.fromXContent(ParsedFilter.java:34)
at org.elasticsearch.client.RestHighLevelClient.lambda$getDefaultNamedXContents$51(RestHighLevelClient.java:2120)
at org.elasticsearch.common.xcontent.NamedXContentRegistry.parseNamedObject(NamedXContentRegistry.java:141)
at org.elasticsearch.common.xcontent.support.AbstractXContentParser.namedObject(AbstractXContentParser.java:415)
at org.elasticsearch.common.xcontent.XContentParserUtils.parseTypedKeysObject(XContentParserUtils.java:153)
at org.elasticsearch.search.aggregations.Aggregations.fromXContent(Aggregations.java:141)
at org.elasticsearch.action.search.SearchResponse.innerFromXContent(SearchResponse.java:292)
at org.elasticsearch.action.search.SearchResponse.fromXContent(SearchResponse.java:251)
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:2053)
at org.elasticsearch.client.RestHighLevelClient.lambda$performRequestAndParseEntity$13(RestHighLevelClient.java:1697)
at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1781)
at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1734)
at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1696)
at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:1092)
at xxx.dao.impl.QueryService.buckets(QueryService.java:425)
at xxx.web.rest.SearchResource.getBuckets(SearchResource.java:95)
at xxx.web.rest.SearchResource$$FastClassBySpringCGLIB$$cec506a1.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
...
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
... 88 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.search.aggregations.metrics.cardinality.ParsedCardinality
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 142 common frames omitted
2019-08-15 09:37:47.420 WARN 472 --- [ XNIO-2 task-1] .m.m.a.ExceptionHandlerExceptionResolver : Resolved exception caused by Handler execution: org.springframework.web.util.NestedServletException: Handler dispatch failed; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/search/aggregations/metrics/cardinality/ParsedCardinality
I cannot think of a reason that would cause besides some classpath issues in your application. The version you are using was compiled correctly and relies on the ParsedCardinality class, but at runtime such class is not there when it needs to be loaded. Given that the high-level client depends on the whole elasticsearch server, you possibly have a version of the client in your classpath, but a different version of elasticsearch which does not support yet the cardinality aggregation. Could you check that please?
The HLRC 6.8.1 depends on Elasticsearch 6.8.1 jar. Your dependencies are overriding that somewhere, that is why it says elasticsearch: 2.4.6 (managed from 6.8.1).
Also, please don't post images of text as they are hardly readable and not searchable. Instead paste the text and format it with </> icon. Check the preview window.
Your application's pom file would be useful to debug this issue.
I am using Spring Boot Starter 1.5.1. It bundles with elasticsearch 2.4.6 jar, that is why 2.4.6 is the managed version. But HLRC 6.8.1 is not smart enough to use the overiding jar. (Strange though, if I lower the Elasticsearch jar to 6.8 instead of 7.x. 6.8.1 HLRC does know to use the 6.8 jar, there is no cardinality error.) So, what is the problem? Latest Spring Boot 2.1.7 is using Elasticsearch 6.4. So, if I upgrade, cardinality should be there. But some other error will happened.
So, I still want to know why overriding jar did not get pick up by 6.8 HLRC correctly when ES jar is 7.x (use 6.8.x ES jar and 6.8.x HLRC jar does not have cardinality error. But when connecting to 7.x ES, some of the aggregation result on 7.X ES is not correct.)
BTW, the image I posted is not a text, it is a window print screen(It is not a text screen). I have no way of formatting it.
I think I have figured out the NoClassDefFoundError problem.
In ES 7.x Lib, there is:
import org.elasticsearch.search.aggregations.metrics.CardinalityAggregationBuilder;
import org.elasticsearch.search.aggregations.metrics.ParsedCardinality;
In ES 6.x lib, it is:
import org.elasticsearch.search.aggregations.metrics.cardinality.CardinalityAggregationBuilder;
import org.elasticsearch.search.aggregations.metrics.cardinality.ParsedCardinality;
So, when using 6.x HLRC, it is expecting
org.elasticsearch.search.aggregations.metrics.cardinality.ParsedCardinality.
But I am using ES 7.x lib which does not have this class, but the
org.elasticsearch.search.aggregations.metrics.ParsedCardinality
Is there any reason for the name change from 6.x to 7.x? I did not see it in the breaking changes note.
I have no easy way of getting around this problem. Lower ES lib to 6.x will have totalHit() as 0 when accessing ES 7.x, since 7.x change the totalHit() to an object with value and relation, 6.x just a long value.
Anyway Elastic.co can change HLRC 6.x lib or 7.x ES lib to make HLRC 6.x compatible with 7.x ES?
Thanks
It is true that the classes changed package in 7.x, but I don't think that is a breaking change. If you use the 6.x client, it depends on elasticsearch 6.x which has the classes in the right place. If you use the 7.x client, it depends on elasticsearch 7.x which has the classes in the new location, still working though.
I suspect that you are using two different versions of client/server libraries.
We said that if you send requests to both 6.x and 7.x, you should keep on using the 6.x client, which depends on 6.x libraries. That should work.
@javanna Thanks for the reply.
I think in my previous reply, I said that " Lower ES lib to 6.x will have totalHit() as 0 when accessing ES 7.x, since 7.x change the totalHit() to an object with value and relation, 6.x just a long value."
So, using both 6.x client and lib does not work. I think this one you can test it out easily.
I see what you mean, the point is that we would never recommend to manually change the libs you use. 6.x works with 6.x, it will give unexpected results when used with 7.x libraries, like you are discovering.
When you have problems with total hits, you are hitting this bug: https://github.com/elastic/elasticsearch/issues/43925 . I would suggest to monitor that issue to see when progress is made on it. Meanwhile, I am afraid you will need to use the 7.x client to talk to 7.x nodes, and the 6.x client to talk to 6.x nodes. IN fact, this is the recommended approach and mixed clusters are really only needed while you are upgrading, which is a temporary situation.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.