After upgrade to elastic search 1.2.1 getting org.elasticsearch.transport.RemoteTransportException: Failed to deserialize response of type [org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse]

Ritesh_Adval · June 11, 2014, 5:01pm

Hi We have upgraded our cluster to elastic search 1.2.1 from 0.90.1, its
running using java 1.7.0.60

We upgraded clients to use 1.2.1 java library and when running it on a
differnt box which has same java 1.7.0.60 I get this exception:

11 Jun 2014 00:38:54,979 INFO [elasticsearch[MN-E
(Ultraverse)][generic][T#1]]
(org.elasticsearch.common.logging.log4j.Log4jESLogger.internalInfo:119) -
[MN-E (Ultraverse)] failed to get node info for
[#transport#-2][gticket1.r6.03c.snaa.gaikai.org][inet[esm2.r6.03c.snaa.gaikai.org/10.9.134.221:9300]],
disconnecting...
org.elasticsearch.transport.RemoteTransportException: Failed to deserialize
response of type
[org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse]
Caused by: org.elasticsearch.transport.TransportSerializationException:
Failed to deserialize response of type
[org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse]
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:148)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:125)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:108)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:89)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IndexOutOfBoundsException: Readable byte limit
exceeded: 8053
at
org.elasticsearch.common.netty.buffer.AbstractChannelBuffer.readByte(AbstractChannelBuffer.java:236)
at
org.elasticsearch.transport.netty.ChannelBufferStreamInput.readByte(ChannelBufferStreamInput.java:132)
at
org.elasticsearch.common.io.stream.StreamInput.readString(StreamInput.java:266)
at
org.elasticsearch.common.io.stream.HandlesStreamInput.readString(HandlesStreamInput.java:61)
at
org.elasticsearch.threadpool.ThreadPool$Info.readFrom(ThreadPool.java:596)
at
org.elasticsearch.threadpool.ThreadPoolInfo.readFrom(ThreadPoolInfo.java:65)
at
org.elasticsearch.threadpool.ThreadPoolInfo.readThreadPoolInfo(ThreadPoolInfo.java:55)
at
org.elasticsearch.action.admin.cluster.node.info.NodeInfo.readFrom(NodeInfo.java:224)
at
org.elasticsearch.action.admin.cluster.node.info.NodeInfo.readNodeInfo(NodeInfo.java:194)
at
org.elasticsearch.action.admin.cluster.node.info.NodesInfoResponse.readFrom(NodesInfoResponse.java:54)
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:146)
... 23 more

Whats wierd is that if I run a single elastic search node on my box which
has java 1.7.0.17 and run the client on the same box, it works fine.

Looks like a jdk issue? Not sure what to do, any suggestions?

Thanks
Ritesh

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7dd193d5-0a76-4420-b1f8-90fc0de0ed2c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ritesh_Adval · June 11, 2014, 7:18pm

Adding client.transport.sniffto true in the client seems to work, not sure
why I need it now when upgrading to 1.2.1.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/6b0783c7-29b2-49b9-9b3d-37c00ff6c451%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Ritesh_Adval · June 12, 2014, 9:42pm

It looks like a regression in 1.2.1 . This is consistently reproducible in
a cluster of 2 or more nodes.

Even if you specify static list of node addresses to connect to, Transport
Client does not work unless you enable sniff?

Can someone take a look and fix?

Thanls
Ritesh

On Wednesday, June 11, 2014 12:18:16 PM UTC-7, Ritesh Adval wrote:

Adding client.transport.sniffto true in the client seems to work, not sure
why I need it now when upgrading to 1.2.1.

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/5da33453-0dc4-4c6a-9fd4-b87fb892115e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

warkolm · June 12, 2014, 10:29pm

If you believe you have found a bug it's best to raise an issue on github.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 13 June 2014 07:42, Ritesh Adval riteshadval@gaikai.com wrote:

It looks like a regression in 1.2.1 . This is consistently reproducible in
a cluster of 2 or more nodes.

Even if you specify static list of node addresses to connect to, Transport
Client does not work unless you enable sniff?

Can someone take a look and fix?

Thanls
Ritesh

On Wednesday, June 11, 2014 12:18:16 PM UTC-7, Ritesh Adval wrote:

Adding client.transport.sniffto true in the client seems to work, not
sure why I need it now when upgrading to 1.2.1.

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5da33453-0dc4-4c6a-9fd4-b87fb892115e%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/5da33453-0dc4-4c6a-9fd4-b87fb892115e%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624aoLSJy9eFdtNvkLLMNX4_evaR08YmM8buDNA%2Bqa4jVkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

maf · June 17, 2014, 8:38am

I have also encountered this, did the debugging and created an issue:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/77d17a29-db6d-44c7-9afd-04d8161fac74%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.