Index corruption and recovery loop


(Vadim Molokovich) #1

Hi
After network connectivity failure we have corrupted index recovery loop in
cluster.
Any help will be greatly appreciated!

*Cluster state: *

elasticsearch version 1.3.0

"cluster_name" : "segments",
"status" : "red",
"timed_out" : false,
"number_of_nodes" : 6,
"number_of_data_nodes" : 6,
"active_primary_shards" : 36,
"active_shards" : 72,
"relocating_shards" : 0,
"initializing_shards" : 5,
"unassigned_shards" : 5
}

Error code:[2014-08-03 16:36:32,219][WARN ][cluster.action.shard ] [sgm-es3] [profiles][3] sending failed shard for [profiles][3], node[ul4j2QmjT1OC885vqCdGZw], [P], s[INITIALIZING], indexUUID [1OFm5O1AQjyn9zPwJqbrsg], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[profiles][3] failed to fetch index version after copying it over]; nested: CorruptIndexException[[profiles][3] Corrupted index [corrupted_C1fjnSSiTn2veXgbo7tREA] caused by: CorruptIndexException[codec footer mismatch: actual footer=1852273495 vs expected footer=-1071082520 (resource: NIOFSIndexInput(path="/usr/local/var/data/elasticsearch/segments/nodes/0/indices/profiles/3/index/_1on_XBloomFilter_0.smy"))]]; ]]*
Shards status:
*

profiles 2 p INITIALIZING 144.76.224.205 sgm-es3
profiles 2 r UNASSIGNED
profiles 0 p INITIALIZING 144.76.96.137 sgm-es1

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/229c6aca-9995-466d-8a79-d570bb05907f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #2