After upgrade to elastic search 1.2.1 getting org.elasticsearch.transport.RemoteTransportException: Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse]


(Ritesh Adval) #1

Hi We have upgraded our cluster to elastic search 1.2.1 from 0.90.1, its
running using java 1.7.0.60

We upgraded clients to use 1.2.1 java library and when running it on a
differnt box which has same java 1.7.0.60 I get this exception:

11 Jun 2014 00:38:54,979 INFO [elasticsearch[MN-E
(Ultraverse)][generic][T#1]]
(org.elasticsearch.common.logging.log4j.Log4jESLogger.internalInfo:119) -
[MN-E (Ultraverse)] failed to get node info for
[#transport#-2][gticket1.r6.03c.snaa.gaikai.org][inet[esm2.r6.03c.snaa.gaikai.org/10.9.134.221:9300]],
disconnecting...
org.elasticsearch.transport.RemoteTransportException: Failed to deserialize
response of type
[org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse]
Caused by: org.elasticsearch.transport.TransportSerializationException:
Failed to deserialize response of type
[org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse]
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:148)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:125)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IndexOutOfBoundsException: Readable byte limit
exceeded: 8053
at
org.elasticsearch.common.netty.buffer.AbstractChannelBuffer.readByte(AbstractChannelBuffer.java:236)
at
org.elasticsearch.transport.netty.ChannelBufferStreamInput.readByte(ChannelBufferStreamInput.java:132)
at
org.elasticsearch.common.io.stream.StreamInput.readString(StreamInput.java:266)
at
org.elasticsearch.common.io.stream.HandlesStreamInput.readString(HandlesStreamInput.java:61)
at
org.elasticsearch.threadpool.ThreadPool$Info.readFrom(ThreadPool.java:596)
at
org.elasticsearch.threadpool.ThreadPoolInfo.readFrom(ThreadPoolInfo.java:65)
at
org.elasticsearch.threadpool.ThreadPoolInfo.readThreadPoolInfo(ThreadPoolInfo.java:55)
at
org.elasticsearch.action.admin.cluster.node.info.NodeInfo.readFrom(NodeInfo.java:224)
at
org.elasticsearch.action.admin.cluster.node.info.NodeInfo.readNodeInfo(NodeInfo.java:194)
at
org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse.readFrom(NodesInfoResponse.java:54)
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:146)
... 23 more

Whats wierd is that if I run a single elastic search node on my box which
has java 1.7.0.17 and run the client on the same box, it works fine.

Looks like a jdk issue? Not sure what to do, any suggestions?

Thanks
Ritesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7dd193d5-0a76-4420-b1f8-90fc0de0ed2c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ritesh Adval) #2

Adding client.transport.sniffto true in the client seems to work, not sure
why I need it now when upgrading to 1.2.1.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6b0783c7-29b2-49b9-9b3d-37c00ff6c451%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ritesh Adval) #3

It looks like a regression in 1.2.1 . This is consistently reproducible in
a cluster of 2 or more nodes.

Even if you specify static list of node addresses to connect to, Transport
Client does not work unless you enable sniff?

Can someone take a look and fix?

Thanls
Ritesh

On Wednesday, June 11, 2014 12:18:16 PM UTC-7, Ritesh Adval wrote:

Adding client.transport.sniffto true in the client seems to work, not sure
why I need it now when upgrading to 1.2.1.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5da33453-0dc4-4c6a-9fd4-b87fb892115e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Mark Walkom) #4

If you believe you have found a bug it's best to raise an issue on github.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 13 June 2014 07:42, Ritesh Adval riteshadval@gaikai.com wrote:

It looks like a regression in 1.2.1 . This is consistently reproducible in
a cluster of 2 or more nodes.

Even if you specify static list of node addresses to connect to, Transport
Client does not work unless you enable sniff?

Can someone take a look and fix?

Thanls
Ritesh

On Wednesday, June 11, 2014 12:18:16 PM UTC-7, Ritesh Adval wrote:

Adding client.transport.sniffto true in the client seems to work, not
sure why I need it now when upgrading to 1.2.1.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5da33453-0dc4-4c6a-9fd4-b87fb892115e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5da33453-0dc4-4c6a-9fd4-b87fb892115e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aoLSJy9eFdtNvkLLMNX4_evaR08YmM8buDNA%2Bqa4jVkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(maf) #5

I have also encountered this, did the debugging and created an issue:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/77d17a29-db6d-44c7-9afd-04d8161fac74%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #6