# Upgraded to 90.6 : RemoteTransportException: Failed to deserialize response of type \[org.elasticsearch.search.query.QuerySearchResult\] **URL:** \ **Category:** Elasticsearch\ **Created:** [November 14, 2013, 3:53pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410 "2013-11-14T15:53:41Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Mark\_Conlin](https://avatars.discourse-cdn.com/v4/letter/m/838e76/32.png) [@Mark\_Conlin](https://discuss.elastic.co/u/Mark_Conlin)\ **Post date:** [November 14, 2013, 3:53pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/1 "2013-11-14T15:53:41Z")
I have a mini cluster of two nodes running 90.6 Both contain data, one is the master. I have written two custom faceting plugins against 90.6. - the first worked against 90.3 and I have upgraded it - the second is brand new and was written against 90.6 Both plugins work well and pass all unit tests (running in memory node during testing). Both work well when only one node of my cluster is running. Both fail when two nodes are running. Clearly transport of query results is not working well, any thoughts on what this might be? Next steps for trouble shooting? _Stack Trace:_ [[A[2013-11-14 10:44:43,533][DEBUG][action.search.type] [90\_6\_node\_1] [v2-20131113][0], node[D13zlQZ4TnyCXNBF6DwR7g], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@44a9bbdb] org.elasticsearch.transport.RemoteTransportException: Failed to deserialize response of type [org.elasticsearch.search.query.QuerySearchResult] Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize response of type [org.elasticsearch.search.query. QuerySearchResult] at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse( MessageChannelHandler.java:148) at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived( MessageChannelHandler.java:125) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel. DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream( DefaultChannelPipeline.java:791) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( Channels.java:296) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. unfoldAndFireMessageReceived(FrameDecoder.java:462) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. callDecode(FrameDecoder.java:443) at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. messageReceived(FrameDecoder.java:303) at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. handleUpstream(SimpleChannelUpstreamHandler.java:70) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. sendUpstream(DefaultChannelPipeline.java:564) at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. sendUpstream(DefaultChannelPipeline.java:559) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( Channels.java:268) at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( Channels.java:255) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read( NioWorker.java:88) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker. process(AbstractNioWorker.java:109) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector. run(AbstractNioSelector.java:312) at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run( AbstractNioWorker.java:90) at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run( NioWorker.java:178) at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run( ThreadRenamingRunnable.java:108) at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run( DeadLockProofWorker.java:42) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. java:615) at java.lang.Thread.run(Thread.java:744) Caused by: java.lang.IndexOutOfBoundsException: Readable byte limit exceeded : 64 at org.elasticsearch.common.netty.buffer.AbstractChannelBuffer.readByte( AbstractChannelBuffer.java:236) at org.elasticsearch.transport.netty.ChannelBufferStreamInput.readByte( ChannelBufferStreamInput.java:132) at org.elasticsearch.common.io.stream.AdapterStreamInput.readByte( AdapterStreamInput.java:35) at org.elasticsearch.common.io.stream.StreamInput.readBoolean(StreamInput. java:267) at org.elasticsearch.search.query.QuerySearchResult.readFrom( QuerySearchResult.java:149) at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse( MessageChannelHandler.java:146) ... 23 more -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Leonardo\_Menezes](https://avatars.discourse-cdn.com/v4/letter/l/b487fb/32.png) [@Leonardo\_Menezes](https://discuss.elastic.co/u/Leonardo_Menezes)\ **Post date:** [November 14, 2013, 4:05pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/2 "2013-11-14T16:05:34Z")
Are both nodes on the same Java version? I have seen errors like that before due to different Java version. On Thu, Nov 14, 2013 at 4:53 PM, Mark Conlin [mark.conlin@gmail.com](mailto:mark.conlin@gmail.com) wrote: > I have a mini cluster of two nodes running 90.6 > Both contain data, one is the master. > > I have written two custom faceting plugins against 90.6. > > - the first worked against 90.3 and I have upgraded it > - the second is brand new and was written against 90.6 > > Both plugins work well and pass all unit tests (running in memory node > during testing). > > Both work well when only one node of my cluster is running. > > Both fail when two nodes are running. > > Clearly transport of query results is not working well, any thoughts on > what this might be? Next steps for trouble shooting? > > _Stack Trace:_ > > [[A[2013-11-14 10:44:43,533][DEBUG][action.search.type] [90 > \_6\_node\_1] [v2-20131113][0], node[D13zlQZ4TnyCXNBF6DwR7g], [P], s[STARTED > ]: Failed to execute [org.elasticsearch.action.search. > SearchRequest@44a9bbdb] > org.elasticsearch.transport.RemoteTransportException: Failed to > deserialize response of type [org.elasticsearch.search.query. > QuerySearchResult] > Caused by: org.elasticsearch.transport.TransportSerializationException: > Failed to deserialize response of type [org.elasticsearch.search.query. > QuerySearchResult] > at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse > (MessageChannelHandler.java:148) > at org.elasticsearch.transport.netty.MessageChannelHandler. > messageReceived(MessageChannelHandler.java:125) > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. > handleUpstream(SimpleChannelUpstreamHandler.java:70) > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > sendUpstream(DefaultChannelPipeline.java:564) > at org.elasticsearch.common.netty.channel. > DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream( > DefaultChannelPipeline.java:791) > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > Channels.java:296) > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > unfoldAndFireMessageReceived(FrameDecoder.java:462) > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > callDecode(FrameDecoder.java:443) > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > messageReceived(FrameDecoder.java:303) > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. > handleUpstream(SimpleChannelUpstreamHandler.java:70) > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > sendUpstream(DefaultChannelPipeline.java:564) > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > sendUpstream(DefaultChannelPipeline.java:559) > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > Channels.java:268) > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > Channels.java:255) > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read( > NioWorker.java:88) > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker. > process(AbstractNioWorker.java:109) > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector. > run(AbstractNioSelector.java:312) > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker. > run(AbstractNioWorker.java:90) > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run( > NioWorker.java:178) > at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run( > ThreadRenamingRunnable.java:108) > at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run > (DeadLockProofWorker.java:42) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. > java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor. > java:615) > at java.lang.Thread.run(Thread.java:744) > Caused by: java.lang.IndexOutOfBoundsException: Readable byte limit > exceeded: 64 > at org.elasticsearch.common.netty.buffer.AbstractChannelBuffer.readByte( > AbstractChannelBuffer.java:236) > at org.elasticsearch.transport.netty.ChannelBufferStreamInput.readByte( > ChannelBufferStreamInput.java:132) > at org.elasticsearch.common.io.stream.AdapterStreamInput.readByte( > AdapterStreamInput.java:35) > at org.elasticsearch.common.io.stream.StreamInput.readBoolean(StreamInput > .java:267) > at org.elasticsearch.search.query.QuerySearchResult.readFrom( > QuerySearchResult.java:149) > at org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse > (MessageChannelHandler.java:146) > ... 23 more > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [November 14, 2013, 4:22pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/3 "2013-11-14T16:22:41Z")
Hey, when running your tests, are you actually spinning up more than one node instance in order to make sure, that at least one node has to communicate remotely? Also you should use a client node or a transport client to execute the queries, in order to make sure everything is working. This ensures that your serialization is working. You will be able to use the same test classes elasticsearch is using internally pretty soon, as those will be provided in an extra jar - this should simplify tests like this a lot and have the benefit of randomized testing. --Alex On Thu, Nov 14, 2013 at 5:05 PM, Leonardo Menezes \< [leonardo.menezess@gmail.com](mailto:leonardo.menezess@gmail.com)\> wrote: > Are both nodes on the same Java version? I have seen errors like that > before due to different Java version. > > On Thu, Nov 14, 2013 at 4:53 PM, Mark Conlin [mark.conlin@gmail.com](mailto:mark.conlin@gmail.com)wrote: > > > I have a mini cluster of two nodes running 90.6 > > Both contain data, one is the master. > > > > I have written two custom faceting plugins against 90.6. > > > > - the first worked against 90.3 and I have upgraded it > > - the second is brand new and was written against 90.6 > > > > Both plugins work well and pass all unit tests (running in memory node > > during testing). > > > > Both work well when only one node of my cluster is running. > > > > Both fail when two nodes are running. > > > > Clearly transport of query results is not working well, any thoughts on > > what this might be? Next steps for trouble shooting? > > > > _Stack Trace:_ > > > > [[A[2013-11-14 10:44:43,533][DEBUG][action.search.type] [90 > > \_6\_node\_1] [v2-20131113][0], node[D13zlQZ4TnyCXNBF6DwR7g], [P], s[STARTED > > ]: Failed to execute [org.elasticsearch.action.search. > > SearchRequest@44a9bbdb] > > org.elasticsearch.transport.RemoteTransportException: Failed to > > deserialize response of type [org.elasticsearch.search.query. > > QuerySearchResult] > > Caused by: org.elasticsearch.transport.TransportSerializationException: > > Failed to deserialize response of type [org.elasticsearch.search.query. > > QuerySearchResult] > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > handleResponse(MessageChannelHandler.java:148) > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > messageReceived(MessageChannelHandler.java:125) > > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. > > handleUpstream(SimpleChannelUpstreamHandler.java:70) > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > sendUpstream(DefaultChannelPipeline.java:564) > > at org.elasticsearch.common.netty.channel. > > DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream( > > DefaultChannelPipeline.java:791) > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > > Channels.java:296) > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > unfoldAndFireMessageReceived(FrameDecoder.java:462) > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > callDecode(FrameDecoder.java:443) > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > messageReceived(FrameDecoder.java:303) > > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. > > handleUpstream(SimpleChannelUpstreamHandler.java:70) > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > sendUpstream(DefaultChannelPipeline.java:564) > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > sendUpstream(DefaultChannelPipeline.java:559) > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > > Channels.java:268) > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > > Channels.java:255) > > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read( > > NioWorker.java:88) > > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker. > > process(AbstractNioWorker.java:109) > > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector > > .run(AbstractNioSelector.java:312) > > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker. > > run(AbstractNioWorker.java:90) > > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run( > > NioWorker.java:178) > > at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run( > > ThreadRenamingRunnable.java:108) > > at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1. > > run(DeadLockProofWorker.java:42) > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor. > > java:1145) > > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor > > .java:615) > > at java.lang.Thread.run(Thread.java:744) > > Caused by: java.lang.IndexOutOfBoundsException: Readable byte limit > > exceeded: 64 > > at org.elasticsearch.common.netty.buffer.AbstractChannelBuffer.readByte( > > AbstractChannelBuffer.java:236) > > at org.elasticsearch.transport.netty.ChannelBufferStreamInput.readByte( > > ChannelBufferStreamInput.java:132) > > at org.elasticsearch.common.io.stream.AdapterStreamInput.readByte( > > AdapterStreamInput.java:35) > > at org.elasticsearch.common.io.stream.StreamInput.readBoolean( > > StreamInput.java:267) > > at org.elasticsearch.search.query.QuerySearchResult.readFrom( > > QuerySearchResult.java:149) > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > handleResponse(MessageChannelHandler.java:146) > > ... 23 more > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Mark\_Conlin](https://avatars.discourse-cdn.com/v4/letter/m/838e76/32.png) [@Mark\_Conlin](https://discuss.elastic.co/u/Mark_Conlin)\ **Post date:** [November 14, 2013, 4:30pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/4 "2013-11-14T16:30:38Z")
I am positive both jars have: Same Java Same es 90.6 code Same custom facet plugin code I use one es and plugins install, and different configs to start the two nodes. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Mark\_Conlin](https://avatars.discourse-cdn.com/v4/letter/m/838e76/32.png) [@Mark\_Conlin](https://discuss.elastic.co/u/Mark_Conlin)\ **Post date:** [November 14, 2013, 4:31pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/5 "2013-11-14T16:31:11Z")
sorry , that should be both nodes have: not jars... On Thursday, November 14, 2013 11:30:38 AM UTC-5, Mark Conlin wrote: > I am positive both jars have: > > Same Java > Same es 90.6 code > Same custom facet plugin code > > I use one es and plugins install, and different configs to start the two > nodes. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Mark\_Conlin](https://avatars.discourse-cdn.com/v4/letter/m/838e76/32.png) [@Mark\_Conlin](https://discuss.elastic.co/u/Mark_Conlin)\ **Post date:** [November 14, 2013, 6:24pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/6 "2013-11-14T18:24:21Z")
Thanks for this, I am now able to make this error occur under unit testing. Clearly, I am not serializing or deserializing my custom facet results or query correctly. Not really sure yet. Mark On Thursday, November 14, 2013 11:22:41 AM UTC-5, Alexander Reelsen wrote: > Hey, > > when running your tests, are you actually spinning up more than one node > instance in order to make sure, that at least one node has to communicate > remotely? Also you should use a client node or a transport client to > execute the queries, in order to make sure everything is working. This > ensures that your serialization is working. > > You will be able to use the same test classes elasticsearch is using > internally pretty soon, as those will be provided in an extra jar - this > should simplify tests like this a lot and have the benefit of randomized > testing. > > --Alex > > On Thu, Nov 14, 2013 at 5:05 PM, Leonardo Menezes \<[leonardo...@gmail.com](mailto:leonardo...@gmail.com)\ > > > wrote: > > > Are both nodes on the same Java version? I have seen errors like that > > before due to different Java version. > > > > On Thu, Nov 14, 2013 at 4:53 PM, Mark Conlin \<[mark....@gmail.com](mailto:mark....@gmail.com)\ > > > > > wrote: > > > > > I have a mini cluster of two nodes running 90.6 > > > Both contain data, one is the master. > > > > > > I have written two custom faceting plugins against 90.6. > > > > > > - the first worked against 90.3 and I have upgraded it > > > - the second is brand new and was written against 90.6 > > > > > > Both plugins work well and pass all unit tests (running in memory node > > > during testing). > > > > > > Both work well when only one node of my cluster is running. > > > > > > Both fail when two nodes are running. > > > > > > Clearly transport of query results is not working well, any thoughts on > > > what this might be? Next steps for trouble shooting? > > > > > > _Stack Trace:_ > > > > > > [[A[2013-11-14 10:44:43,533][DEBUG][action.search.type] [90 > > > \_6\_node\_1] [v2-20131113][0], node[D13zlQZ4TnyCXNBF6DwR7g], [P], s[ > > > STARTED]: Failed to execute [org.elasticsearch.action.search. > > > SearchRequest@44a9bbdb] > > > org.elasticsearch.transport.RemoteTransportException: Failed to > > > deserialize response of type [org.elasticsearch.search.query. > > > QuerySearchResult] > > > Caused by: org.elasticsearch.transport.TransportSerializationException: > > > Failed to deserialize response of type [org.elasticsearch.search.query. > > > QuerySearchResult] > > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > > handleResponse(MessageChannelHandler.java:148) > > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > > messageReceived(MessageChannelHandler.java:125) > > > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. > > > handleUpstream(SimpleChannelUpstreamHandler.java:70) > > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > > sendUpstream(DefaultChannelPipeline.java:564) > > > at org.elasticsearch.common.netty.channel. > > > DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream( > > > DefaultChannelPipeline.java:791) > > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > > > Channels.java:296) > > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > > unfoldAndFireMessageReceived(FrameDecoder.java:462) > > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > > callDecode(FrameDecoder.java:443) > > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > > messageReceived(FrameDecoder.java:303) > > > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler. > > > handleUpstream(SimpleChannelUpstreamHandler.java:70) > > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > > sendUpstream(DefaultChannelPipeline.java:564) > > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > > sendUpstream(DefaultChannelPipeline.java:559) > > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > > > Channels.java:268) > > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived( > > > Channels.java:255) > > > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read( > > > NioWorker.java:88) > > > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker. > > > process(AbstractNioWorker.java:109) > > > at org.elasticsearch.common.netty.channel.socket.nio. > > > AbstractNioSelector.run(AbstractNioSelector.java:312) > > > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker. > > > run(AbstractNioWorker.java:90) > > > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run( > > > NioWorker.java:178) > > > at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run( > > > ThreadRenamingRunnable.java:108) > > > at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1. > > > run(DeadLockProofWorker.java:42) > > > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor > > > .java:1145) > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > > ThreadPoolExecutor.java:615) > > > at java.lang.Thread.run(Thread.java:744) > > > Caused by: java.lang.IndexOutOfBoundsException: Readable byte limit > > > exceeded: 64 > > > at org.elasticsearch.common.netty.buffer.AbstractChannelBuffer.readByte > > > (AbstractChannelBuffer.java:236) > > > at org.elasticsearch.transport.netty.ChannelBufferStreamInput.readByte( > > > ChannelBufferStreamInput.java:132) > > > at org.elasticsearch.common.io.stream.AdapterStreamInput.readByte( > > > AdapterStreamInput.java:35) > > > at org.elasticsearch.common.io.stream.StreamInput.readBoolean( > > > StreamInput.java:267) > > > at org.elasticsearch.search.query.QuerySearchResult.readFrom( > > > QuerySearchResult.java:149) > > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > > handleResponse(MessageChannelHandler.java:146) > > > ... 23 more > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Mark\_Conlin](https://avatars.discourse-cdn.com/v4/letter/m/838e76/32.png) [@Mark\_Conlin](https://discuss.elastic.co/u/Mark_Conlin)\ **Post date:** [November 15, 2013, 4:10pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/7 "2013-11-15T16:10:33Z")
SOLVED : It was bad writeTo / readFrom in my plugin. Lessons learned : - you really should unit test and integration test with multiple nodes - if your master node has data, and your client app doesnt check results for shard failures and failure message and simply consumes data from the shards that worked, you wont see intra-node errors like this - elasticsearch logs DEBUG for these errors On Thursday, November 14, 2013 1:24:21 PM UTC-5, Mark Conlin wrote: > Thanks for this, I am now able to make this error occur under unit > testing. > Clearly, I am not serializing or deserializing my custom facet results or > query correctly. Not really sure yet. > > Mark > > On Thursday, November 14, 2013 11:22:41 AM UTC-5, Alexander Reelsen wrote: > > > Hey, > > > > when running your tests, are you actually spinning up more than one node > > instance in order to make sure, that at least one node has to communicate > > remotely? Also you should use a client node or a transport client to > > execute the queries, in order to make sure everything is working. This > > ensures that your serialization is working. > > > > You will be able to use the same test classes elasticsearch is using > > internally pretty soon, as those will be provided in an extra jar - this > > should simplify tests like this a lot and have the benefit of randomized > > testing. > > > > --Alex > > > > On Thu, Nov 14, 2013 at 5:05 PM, Leonardo Menezes [leonardo...@gmail.com](mailto:leonardo...@gmail.com)wrote: > > > > > Are both nodes on the same Java version? I have seen errors like that > > > before due to different Java version. > > > > > > On Thu, Nov 14, 2013 at 4:53 PM, Mark Conlin [mark....@gmail.com](mailto:mark....@gmail.com) wrote: > > > > > > > I have a mini cluster of two nodes running 90.6 > > > > Both contain data, one is the master. > > > > > > > > I have written two custom faceting plugins against 90.6. > > > > > > > > - the first worked against 90.3 and I have upgraded it > > > > - the second is brand new and was written against 90.6 > > > > > > > > Both plugins work well and pass all unit tests (running in memory node > > > > during testing). > > > > > > > > Both work well when only one node of my cluster is running. > > > > > > > > Both fail when two nodes are running. > > > > > > > > Clearly transport of query results is not working well, any thoughts on > > > > what this might be? Next steps for trouble shooting? > > > > > > > > _Stack Trace:_ > > > > > > > > [[A[2013-11-14 10:44:43,533][DEBUG][action.search.type] [90 > > > > \_6\_node\_1] [v2-20131113][0], node[D13zlQZ4TnyCXNBF6DwR7g], [P], s[ > > > > STARTED]: Failed to execute [org.elasticsearch.action.search. > > > > SearchRequest@44a9bbdb] > > > > org.elasticsearch.transport.RemoteTransportException: Failed to > > > > deserialize response of type [org.elasticsearch.search.query. > > > > QuerySearchResult] > > > > Caused by: org.elasticsearch.transport.TransportSerializationException: > > > > Failed to deserialize response of type [org.elasticsearch.search.query. > > > > QuerySearchResult] > > > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > > > handleResponse(MessageChannelHandler.java:148) > > > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > > > messageReceived(MessageChannelHandler.java:125) > > > > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler > > > > .handleUpstream(SimpleChannelUpstreamHandler.java:70) > > > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > > > sendUpstream(DefaultChannelPipeline.java:564) > > > > at org.elasticsearch.common.netty.channel. > > > > DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream( > > > > DefaultChannelPipeline.java:791) > > > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived > > > > (Channels.java:296) > > > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > > > unfoldAndFireMessageReceived(FrameDecoder.java:462) > > > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > > > callDecode(FrameDecoder.java:443) > > > > at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder. > > > > messageReceived(FrameDecoder.java:303) > > > > at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler > > > > .handleUpstream(SimpleChannelUpstreamHandler.java:70) > > > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > > > sendUpstream(DefaultChannelPipeline.java:564) > > > > at org.elasticsearch.common.netty.channel.DefaultChannelPipeline. > > > > sendUpstream(DefaultChannelPipeline.java:559) > > > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived > > > > (Channels.java:268) > > > > at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived > > > > (Channels.java:255) > > > > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read( > > > > NioWorker.java:88) > > > > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker > > > > .process(AbstractNioWorker.java:109) > > > > at org.elasticsearch.common.netty.channel.socket.nio. > > > > AbstractNioSelector.run(AbstractNioSelector.java:312) > > > > at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker > > > > .run(AbstractNioWorker.java:90) > > > > at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run( > > > > NioWorker.java:178) > > > > at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run( > > > > ThreadRenamingRunnable.java:108) > > > > at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1. > > > > run(DeadLockProofWorker.java:42) > > > > at java.util.concurrent.ThreadPoolExecutor.runWorker( > > > > ThreadPoolExecutor.java:1145) > > > > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > > > > ThreadPoolExecutor.java:615) > > > > at java.lang.Thread.run(Thread.java:744) > > > > Caused by: java.lang.IndexOutOfBoundsException: Readable byte limit > > > > exceeded: 64 > > > > at org.elasticsearch.common.netty.buffer.AbstractChannelBuffer. > > > > readByte(AbstractChannelBuffer.java:236) > > > > at org.elasticsearch.transport.netty.ChannelBufferStreamInput.readByte > > > > (ChannelBufferStreamInput.java:132) > > > > at org.elasticsearch.common.io.stream.AdapterStreamInput.readByte( > > > > AdapterStreamInput.java:35) > > > > at org.elasticsearch.common.io.stream.StreamInput.readBoolean( > > > > StreamInput.java:267) > > > > at org.elasticsearch.search.query.QuerySearchResult.readFrom( > > > > QuerySearchResult.java:149) > > > > at org.elasticsearch.transport.netty.MessageChannelHandler. > > > > handleResponse(MessageChannelHandler.java:146) > > > > ... 23 more > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![bsarkar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bsarkar/32/42967_2.png) [@bsarkar](https://discuss.elastic.co/u/bsarkar)\ **Post date:** [November 29, 2014, 12:58pm UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/8 "2014-11-29T12:58:38Z")
Hi Mark. I am running into the same issue with custom aggregations. After reading your reply, I found a fault in my readFrom() method too and I fixed it but it still did not solve the problem. Did you need to fix anything else? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:47am UTC](https://discuss.elastic.co/t/upgraded-to-90-6-remotetransportexception-failed-to-deserialize-response-of-type-org-elasticsearch-search-query-querysearchresult/14410/9 "2017-07-06T00:47:05Z")