Message not fully read (response) for SearchServiceTransportAction issue after upgrading to 0.90.0

Hi All,

Sorry for this noob question.

My index has 3 shards and 1 replica. There are 3 nodes(es version 0.90.0).
When I do the query searching with search type DFS_QUERY_THEN_FETCH on some
special fields,
there is a warning 'Message not fully read (response)'. Some of the results
are missing.
The results for subsequent requests keep changing. I can see that there are
3 versions for the search result.

Server log are as followings :
[2013-05-09 03:43:13,489][INFO ][index.shard.service ] [Grand
Director] [i_product_ywk_5][1] updating refresh_interval from [-1] to [1s]
[2013-05-09 03:43:13,489][INFO ][index.shard.service ] [Grand
Director] [i_product_ywk_5][0] updating refresh_interval from [-1] to [1s]
[2013-05-09 03:44:50,568][INFO ][action.admin.cluster.node.shutdown] [Grand
Director] shutting down in [200ms]
[2013-05-09 03:44:50,771][INFO ][action.admin.cluster.node.shutdown] [Grand
Director] initiating requested shutdown...
[2013-05-09 03:44:50,771][INFO ][node ] [Grand
Director] {0.90.0}[8128]: stopping ...
[2013-05-09 03:44:50,823][INFO ][node ] [Grand
Director] {0.90.0}[8128]: stopped
[2013-05-09 03:44:50,823][INFO ][node ] [Grand
Director] {0.90.0}[8128]: closing ...
[2013-05-09 03:44:50,833][INFO ][node ] [Grand
Director] {0.90.0}[8128]: closed
[2013-05-09 03:45:35,022][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: initializing ...
[2013-05-09 03:45:35,031][INFO ][plugins ] [Baron
Strucker] loaded [], sites []
[2013-05-09 03:45:37,691][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: initialized
[2013-05-09 03:45:37,692][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: starting ...
[2013-05-09 03:45:37,828][INFO ][transport ] [Baron
Strucker] bound_address {inet[/0.0.0.0:9300]}, publish_address
{inet[/10.1.4.220:9300]}
[2013-05-09 03:45:40,933][INFO ][cluster.service ] [Baron
Strucker] detected_master [Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]], added
{[Puishannt][9ogJ5QcyTHiWhMe-mJCPkQ][inet[/10.1.4.221:9300]],[Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]],}, reason:
zen-disco-receive(from master [[Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]]])
[2013-05-09 03:45:41,031][INFO ][discovery ] [Baron
Strucker] _testinges.0.90.0/THfT5ZzSRPai-PV9bLaP_Q
[2013-05-09 03:45:41,042][INFO ][http ] [Baron
Strucker] bound_address {inet[/0.0.0.0:9200]}, publish_address
{inet[/10.1.4.220:9200]}
[2013-05-09 03:45:41,042][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: started
[2013-05-09 03:46:18,292][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [68] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@28fa1b85,
error [false], resetting
[2013-05-09 03:46:18,304][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [67] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@1441e2aa,
error [false], resetting
[2013-05-09 03:46:18,391][DEBUG][action.search.type ] [Baron
Strucker] [i_product_ys_5][2], node[EtCRx4NOTwuI3bHAn8IGRg], [P],
s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@71881149]
org.elasticsearch.transport.RemoteTransportException: Failed to deserialize
response of type [org.elasticsearch.search.dfs.DfsSearchResult]
Caused by: org.elasticsearch.transport.TransportSerializationException:
Failed to deserialize response of type
[org.elasticsearch.search.dfs.DfsSearchResult]
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:150)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:127)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Expected handle header, got [8]
at
org.elasticsearch.common.io.stream.HandlesStreamInput.readString(HandlesStreamInput.java:65)
at
org.elasticsearch.search.dfs.DfsSearchResult.readFrom(DfsSearchResult.java:140)
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:148)
... 23 more
[2013-05-09 03:46:18,399][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [70] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@326d234c,
error [false], resetting

I found this issue after upgrading the elasticsearch version from 0.19.10
to 0.90.0.
I have no idea why this happens in 0.90.0.
I've verified that only multiple shards will have this problem. Single
shard works fine.
Also, it happens only for my client with JAVA API.

I've tried to re-produce this issue at

For my client and server setting,
The client es jar version is :

org.elasticsearch
elasticsearch
0.90.0

The es servers are using 0.90.0.
ref : http://www.elasticsearch.org/downloads/0-90-0/
All the java versions on the clusters are the same.

Can I ask is there any problem for my settings?

Many thanks.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Have you seen this issue?

Very likely it is related.

Cheers,

Ivan

On Thu, May 9, 2013 at 3:33 AM, keung2640@gmail.com wrote:

Hi All,

Sorry for this noob question.

My index has 3 shards and 1 replica. There are 3 nodes(es version 0.90.0).
When I do the query searching with search type DFS_QUERY_THEN_FETCH on
some special fields,
there is a warning 'Message not fully read (response)'. Some of the
results are missing.
The results for subsequent requests keep changing. I can see that there
are 3 versions for the search result.

Server log are as followings :
[2013-05-09 03:43:13,489][INFO ][index.shard.service ] [Grand
Director] [i_product_ywk_5][1] updating refresh_interval from [-1] to [1s]
[2013-05-09 03:43:13,489][INFO ][index.shard.service ] [Grand
Director] [i_product_ywk_5][0] updating refresh_interval from [-1] to [1s]
[2013-05-09 03:44:50,568][INFO ][action.admin.cluster.node.shutdown]
[Grand Director] shutting down in [200ms]
[2013-05-09 03:44:50,771][INFO ][action.admin.cluster.node.shutdown]
[Grand Director] initiating requested shutdown...
[2013-05-09 03:44:50,771][INFO ][node ] [Grand
Director] {0.90.0}[8128]: stopping ...
[2013-05-09 03:44:50,823][INFO ][node ] [Grand
Director] {0.90.0}[8128]: stopped
[2013-05-09 03:44:50,823][INFO ][node ] [Grand
Director] {0.90.0}[8128]: closing ...
[2013-05-09 03:44:50,833][INFO ][node ] [Grand
Director] {0.90.0}[8128]: closed
[2013-05-09 03:45:35,022][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: initializing ...
[2013-05-09 03:45:35,031][INFO ][plugins ] [Baron
Strucker] loaded , sites
[2013-05-09 03:45:37,691][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: initialized
[2013-05-09 03:45:37,692][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: starting ...
[2013-05-09 03:45:37,828][INFO ][transport ] [Baron
Strucker] bound_address {inet[/0.0.0.0:9300]}, publish_address {inet[/
10.1.4.220:9300]}
[2013-05-09 03:45:40,933][INFO ][cluster.service ] [Baron
Strucker] detected_master [Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]], added
{[Puishannt][9ogJ5QcyTHiWhMe-mJCPkQ][inet[/10.1.4.221:9300]],[Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]],}, reason:
zen-disco-receive(from master [[Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]]])
[2013-05-09 03:45:41,031][INFO ][discovery ] [Baron
Strucker] _testinges.0.90.0/THfT5ZzSRPai-PV9bLaP_Q
[2013-05-09 03:45:41,042][INFO ][http ] [Baron
Strucker] bound_address {inet[/0.0.0.0:9200]}, publish_address {inet[/
10.1.4.220:9200]}
[2013-05-09 03:45:41,042][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: started
[2013-05-09 03:46:18,292][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [68] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@28fa1b85,
error [false], resetting
[2013-05-09 03:46:18,304][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [67] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@1441e2aa,
error [false], resetting
[2013-05-09 03:46:18,391][DEBUG][action.search.type ] [Baron
Strucker] [i_product_ys_5][2], node[EtCRx4NOTwuI3bHAn8IGRg], [P],
s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@71881149]
org.elasticsearch.transport.RemoteTransportException: Failed to
deserialize response of type [org.elasticsearch.search.dfs.DfsSearchResult]
Caused by: org.elasticsearch.transport.TransportSerializationException:
Failed to deserialize response of type
[org.elasticsearch.search.dfs.DfsSearchResult]
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:150)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:127)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Expected handle header, got [8]
at
org.elasticsearch.common.io.stream.HandlesStreamInput.readString(HandlesStreamInput.java:65)
at
org.elasticsearch.search.dfs.DfsSearchResult.readFrom(DfsSearchResult.java:140)
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:148)
... 23 more
[2013-05-09 03:46:18,399][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [70] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@326d234c,
error [false], resetting

I found this issue after upgrading the elasticsearch version from 0.19.10
to 0.90.0.
I have no idea why this happens in 0.90.0.
I've verified that only multiple shards will have this problem. Single
shard works fine.
Also, it happens only for my client with JAVA API.

I've tried to re-produce this issue at
Shooting Message not fully read (response) for handler org.elasticsearch.search.action.SearchServiceTransportAction · GitHub

For my client and server setting,
The client es jar version is :

org.elasticsearch
elasticsearch
0.90.0

The es servers are using 0.90.0.
ref : Elasticsearch Platform — Find real-time answers at scale | Elastic
All the java versions on the clusters are the same.

Can I ask is there any problem for my settings?

Many thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi Ivan,

Thanks for your info and reference. Let me take a look at it.

Ivan Brusic於 2013年5月10日星期五UTC+8上午3時06分20秒寫道:

Have you seen this issue?
DFS modes can cause undefined behaviour in 0.90 · Issue #3012 · elastic/elasticsearch · GitHub

Very likely it is related.

Cheers,

Ivan

On Thu, May 9, 2013 at 3:33 AM, <keun...@gmail.com <javascript:>> wrote:

Hi All,

Sorry for this noob question.

My index has 3 shards and 1 replica. There are 3 nodes(es version
0.90.0).
When I do the query searching with search type DFS_QUERY_THEN_FETCH on
some special fields,
there is a warning 'Message not fully read (response)'. Some of the
results are missing.
The results for subsequent requests keep changing. I can see that there
are 3 versions for the search result.

Server log are as followings :
[2013-05-09 03:43:13,489][INFO ][index.shard.service ] [Grand
Director] [i_product_ywk_5][1] updating refresh_interval from [-1] to [1s]
[2013-05-09 03:43:13,489][INFO ][index.shard.service ] [Grand
Director] [i_product_ywk_5][0] updating refresh_interval from [-1] to [1s]
[2013-05-09 03:44:50,568][INFO ][action.admin.cluster.node.shutdown]
[Grand Director] shutting down in [200ms]
[2013-05-09 03:44:50,771][INFO ][action.admin.cluster.node.shutdown]
[Grand Director] initiating requested shutdown...
[2013-05-09 03:44:50,771][INFO ][node ] [Grand
Director] {0.90.0}[8128]: stopping ...
[2013-05-09 03:44:50,823][INFO ][node ] [Grand
Director] {0.90.0}[8128]: stopped
[2013-05-09 03:44:50,823][INFO ][node ] [Grand
Director] {0.90.0}[8128]: closing ...
[2013-05-09 03:44:50,833][INFO ][node ] [Grand
Director] {0.90.0}[8128]: closed
[2013-05-09 03:45:35,022][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: initializing ...
[2013-05-09 03:45:35,031][INFO ][plugins ] [Baron
Strucker] loaded , sites
[2013-05-09 03:45:37,691][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: initialized
[2013-05-09 03:45:37,692][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: starting ...
[2013-05-09 03:45:37,828][INFO ][transport ] [Baron
Strucker] bound_address {inet[/0.0.0.0:9300]}, publish_address {inet[/
10.1.4.220:9300]}
[2013-05-09 03:45:40,933][INFO ][cluster.service ] [Baron
Strucker] detected_master [Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]], added
{[Puishannt][9ogJ5QcyTHiWhMe-mJCPkQ][inet[/10.1.4.221:9300]],[Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]],}, reason:
zen-disco-receive(from master [[Corbo,
Adrian][EtCRx4NOTwuI3bHAn8IGRg][inet[/10.1.4.222:9300]]])
[2013-05-09 03:45:41,031][INFO ][discovery ] [Baron
Strucker] _testinges.0.90.0/THfT5ZzSRPai-PV9bLaP_Q
[2013-05-09 03:45:41,042][INFO ][http ] [Baron
Strucker] bound_address {inet[/0.0.0.0:9200]}, publish_address {inet[/
10.1.4.220:9200]}
[2013-05-09 03:45:41,042][INFO ][node ] [Baron
Strucker] {0.90.0}[8440]: started
[2013-05-09 03:46:18,292][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [68] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@28fa1b85,
error [false], resetting
[2013-05-09 03:46:18,304][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [67] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@1441e2aa,
error [false], resetting
[2013-05-09 03:46:18,391][DEBUG][action.search.type ] [Baron
Strucker] [i_product_ys_5][2], node[EtCRx4NOTwuI3bHAn8IGRg], [P],
s[STARTED]: Failed to execute
[org.elasticsearch.action.search.SearchRequest@71881149]
org.elasticsearch.transport.RemoteTransportException: Failed to
deserialize response of type [org.elasticsearch.search.dfs.DfsSearchResult]
Caused by: org.elasticsearch.transport.TransportSerializationException:
Failed to deserialize response of type
[org.elasticsearch.search.dfs.DfsSearchResult]
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:150)
at
org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:127)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at
org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at
org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at
org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at
org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at
org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88)
at
org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at
org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at
org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Expected handle header, got [8]
at
org.elasticsearch.common.io.stream.HandlesStreamInput.readString(HandlesStreamInput.java:65)
at
org.elasticsearch.search.dfs.DfsSearchResult.readFrom(DfsSearchResult.java:140)
at
org.elasticsearch.transport.netty.MessageChannelHandler.handleResponse(MessageChannelHandler.java:148)
... 23 more
[2013-05-09 03:46:18,399][WARN ][transport.netty ] [Baron
Strucker] Message not fully read (response) for [70] handler
org.elasticsearch.search.action.SearchServiceTransportAction$1@326d234c,
error [false], resetting

I found this issue after upgrading the elasticsearch version from 0.19.10
to 0.90.0.
I have no idea why this happens in 0.90.0.
I've verified that only multiple shards will have this problem. Single
shard works fine.
Also, it happens only for my client with JAVA API.

I've tried to re-produce this issue at
Shooting Message not fully read (response) for handler org.elasticsearch.search.action.SearchServiceTransportAction · GitHub

For my client and server setting,
The client es jar version is :

org.elasticsearch
elasticsearch
0.90.0

The es servers are using 0.90.0.
ref : Elasticsearch Platform — Find real-time answers at scale | Elastic
All the java versions on the clusters are the same.

Can I ask is there any problem for my settings?

Many thanks.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi All,

I've just found that there are some fixes for related issues in
elasticsearch 0.90.1.
DFS modes can cause undefined behaviour in 0.90 #3012


NPE when using java client with DFS_QUERY_THEN_FETCH #3008

But I can still reproduce this issue after upgrading from 0.90.0 to 0.90.1.

May I know anything I can do for this?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi All,

I've just found that there are some fixes for related issues in
elasticsearch 0.90.1.
DFS modes can cause undefined behaviour in 0.90 #3012


NPE when using java client with DFS_QUERY_THEN_FETCH #3008

But I can still reproduce this issue after upgrading from 0.90.0 to 0.90.1.

May I know anything I can do for this?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hi All,

I've just found that there are some fixes for related issues in
elasticsearch 0.90.1.
DFS modes can cause undefined behaviour in 0.90 #3012


NPE when using java client with DFS_QUERY_THEN_FETCH #3008

But I can still reproduce this issue after upgrading from 0.90.0 to 0.90.1.

May I know anything I can do for this?

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.