Hi, guys:
After I restart the cluster, it became to red, and the cluster cluster
stayed at the below status for long time, unassiged shards and initializing
shards stayed at 26 and 24 for long time, no change.
{
"cluster_name" : "es-test-cluster-1",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 20,
"number_of_data_nodes" : 15,
"active_primary_shards" : 196,
"active_shards" : 350,
"relocating_shards" : 0,
"initializing_shards" : 24,
"unassigned_shards" : 26
}
And I captured some logs from master node:
[2013-02-21 17:30:19,072][DEBUG][cluster.action.shard ] [xseed023.kdev]
Applying failed shard [nelo2-log-2013-02-26][11],
node[BRrHHP-iSDCDkdcNrQrgOA], [R], s[INITIALIZING], reason [Failed to start
shard, message [RecoveryFailedException[[nelo2-log-2013-02-26][11]:
Recovery failed from
[xseed040.kdev][FLPZtxFSQA-1uaYQDyiKMg][inet[/10.96.250.230:9300]]{master=false}
into
[xseed033.kdev][BRrHHP-iSDCDkdcNrQrgOA][inet[/10.96.250.223:9300]]{master=false}];
nested: RemoteTransportException[Failed to deserialize exception response
from stream]; nested: TransportSerializationException[Failed to deserialize
exception response from stream]; nested:
StreamCorruptedException[unexpected end of block data]; ]]
[2013-02-21 17:30:19,073][TRACE][cluster.service ] [xseed023.kdev]
cluster state updated:
version [123], source [shard-failed ([nelo2-log-2013-02-26][11],
node[BRrHHP-iSDCDkdcNrQrgOA], [R], s[INITIALIZING]), reason [Failed to
start shard, message [RecoveryFailedException[[nelo2-log-2013-02-26][11]:
Recovery failed from
[xseed040.kdev][FLPZtxFSQA-1uaYQDyiKMg][inet[/10.96.250.230:9300]]{master=false}
into
[xseed033.kdev][BRrHHP-iSDCDkdcNrQrgOA][inet[/10.96.250.223:9300]]{master=false}];
nested: RemoteTransportException[Failed to deserialize exception response
from stream]; nested: TransportSerializationException[Failed to deserialize
exception response from stream]; nested:
StreamCorruptedException[unexpected end of block data]; ]]]
[2013-02-21 17:30:19,157][DEBUG][gateway.local ] [xseed023.kdev]
[nelo2-log-2013-02-19][18]: throttling allocation
[[nelo2-log-2013-02-19][18], node[null], [R], s[UNASSIGNED]] to
[[xseed030.kdev][JlHMNsATS52ifplzd-lRJw][inet[/10.96.250.220:9300]]{master=false}]
in order to reuse its unallocated persistent store with total_size [3.6gb]
By the way, before restart, the master is node A, but after restart,
the master become to another node B. I'm not sure if it will cause the
problem or not.
Does anyone know how to bring the cluster to green?
Thank you very much!
-Regards-
-Jackie-
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.