Transport client: java.lang.NoSuchMethodError: io.netty.buffer.CompositeByteBuf.addComponents

Elastic Stack Elasticsearch
#1

Hi,
I am running a Spark application using elasticsearch-spark-13_2.11:5.2.2 and I wanted to include also org.elasticsearch.client.transport:5.2.2 but there seems to be a dependency issue. This only arises when I use spark-submit against my Spark-Cluster (1.5.2), running locally works fine.

Exception in thread "elasticsearch[_client_][management][T#1]" java.lang.NoSuchMethodError: io.netty.buffer.CompositeByteBuf.addComponents(ZLjava/lang/Iterable;)Lio/netty/buffer/CompositeByteBuf;
	at org.elasticsearch.transport.netty4.Netty4Utils.toByteBuf(Netty4Utils.java:78)
	at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:422)
	at org.elasticsearch.transport.netty4.Netty4Transport.sendMessage(Netty4Transport.java:93)
	at org.elasticsearch.transport.TcpTransport.internalSendMessage(TcpTransport.java:1058)
	at org.elasticsearch.transport.TcpTransport.sendRequestToChannel(TcpTransport.java:1040)
	at org.elasticsearch.transport.TcpTransport.executeHandshake(TcpTransport.java:1555)
	at org.elasticsearch.transport.TcpTransport.openConnection(TcpTransport.java:502)
	at org.elasticsearch.transport.TcpTransport.connectToNode(TcpTransport.java:460)
	at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:318)
	at org.elasticsearch.client.transport.TransportClientNodesService$SniffNodesSampler$1.run(TransportClientNodesService.java:488)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:527)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
#2

@larghir I seem to remember Spark having version conflicts with the versions of Netty that the transport client makes use of. Do you specifically need the transport client for your workflow? If so, have you taken a look at our official rest client, or thought about using the existing rest client available in ES-Hadoop?

#4

Thanks @james.baiera, I will take a look at the rest client

#5

It's actually (not) funny, there seems to be a dependency conflict with httpcore as well.
Using Elasticsearch's RestClient:

Exception in thread "pool-1-thread-1" java.lang.NoSuchMethodError: org.apache.http.util.Asserts.check(ZLjava/lang/String;Ljava/lang/Object;)V
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:313)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	at java.lang.Thread.run(Thread.java:745)
Exception in thread "pool-2-thread-1" java.lang.NoSuchMethodError: org.apache.http.util.Asserts.check(ZLjava/lang/String;Ljava/lang/Object;)V
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:313)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:192)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	at java.lang.Thread.run(Thread.java:745)
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.http.util.Asserts.check(ZLjava/lang/String;Ljava/lang/Object;)V
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90)
	at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClient.execute(CloseableHttpAsyncClient.java:68)
	at org.elasticsearch.client.RestClient.performRequestAsync(RestClient.java:300)
	at org.elasticsearch.client.RestClient.performRequestAsync(RestClient.java:290)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:211)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:184)
	at org.elasticsearch.client.RestClient.performRequest(RestClient.java:146)
#6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.