# Cluster health RED, UNASSIGNED shards from CLUSTER\_RECOVERED
**URL:** https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389
**Category:** Elasticsearch
**Created:** [April 25, 2018, 2:04am UTC](https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389 "2018-04-25T02:04:18Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@bistaumanga](https://discuss.elastic.co/u/bistaumanga)
#### Post date: [April 25, 2018, 2:04am UTC](https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389/1 "2018-04-25T02:04:19Z")
I have a elasticsearch with 3 nodes, first one is master, and all are data nodes. I create a snapshot of first node and created new vm in different data center (backed by openstack), and copied the elasticsearch data directory. I can start the elasticsearch, access kibana, but some data are UNASSIGNED .
Here are some of the output from few commands i tried.
```
curl -XGET http://search01:9200/_cat/shards | grep UNASSIGNED | wc -l
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 18478 100 18478 0 0 61492 0 --:--:-- --:--:-- --:--:-- 61593
58
ubuntu@search01:~$ curl -XGET http://localhost:9200/_cluster/allocation/explain?pretty
{
"index" : "mmkg-doc-nst02-000003",
"shard" : 0,
"primary" : true,
"current_state" : "unassigned",
"unassigned_info" : {
"reason" : "CLUSTER_RECOVERED",
"at" : "2018-04-25T01:48:14.631Z",
"last_allocation_status" : "no_valid_shard_copy"
},
"can_allocate" : "no_valid_shard_copy",
"allocate_explanation" : "cannot allocate because all found copies of the shard are either stale or corrupt",
"node_allocation_decisions" : [
{
"node_id" : "-AMMeohmQKemcPtvHWwoLQ",
"node_name" : "search01",
"transport_address" : "search01:9300",
"node_decision" : "no",
"store" : {
"found" : false
}
},
{
"node_id" : "qoFEyZQcTh2ppPpO23uB0w",
"node_name" : "search03",
"transport_address" : "search03:9300",
"node_decision" : "no",
"store" : {
"in_sync" : false,
"allocation_id" : "sHfaufsdRRKf0jYi3vNf_Q"
}
},
{
"node_id" : "vyeFBQUOSterpgmlCicWVg",
"node_name" : "search02",
"transport_address" : "search02:9300",
"node_decision" : "no",
"store" : {
"in_sync" : false,
"allocation_id" : "KX5PPj8yTHyJG-UE6_JaLA"
}
}
]
}
```
How to recover the data?
Note: I replaced the ips with hostname in "transport\_address" : "search02:9300" for security.
---
### Author:  [@JKhondhu](https://discuss.elastic.co/u/JKhondhu)
#### Post date: [April 26, 2018, 8:38pm UTC](https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389/2 "2018-04-26T20:38:18Z")
> [@bistaumanga](#):
>
> "allocate\_explanation" : "cannot allocate because all found copies of the shard are either stale or corrupt",
I take it you took a VM snapshot and not an elasticsearch snapshot as per here: [Snapshot module | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html)
If so then see the above `allocate_explanation` reason. If your first cluster is alive then seek to take a full cluster `snapshot` and `restore` that into the other DC cluster.
---
### Author:  [@bistaumanga](https://discuss.elastic.co/u/bistaumanga)
#### Post date: [April 26, 2018, 11:27pm UTC](https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389/3 "2018-04-26T23:27:52Z")
Hi @JKhondhu, I took VM snapshot of one of the 3 node (master node), and transfered data manually from elasticsearch data directory. I just need to move this one node.
---
### Author:  [@JKhondhu](https://discuss.elastic.co/u/JKhondhu)
#### Post date: [April 27, 2018, 6:56am UTC](https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389/4 "2018-04-27T06:56:37Z")
@bistaumanga
Yeah, VM snapshots of the vm it self are cool but as the Elasticsearch shards (Lucene segments) reside on the file system as immutable objects, a VM snapshot will not capture the segments that are being updated accordingly. Therefor you have the explanation of `either stale or corrupt`
If your first cluster is alive and kicking then seek to take a full cluster snapshot and restore that into the other DC cluster.
---
### Author:  [@bistaumanga](https://discuss.elastic.co/u/bistaumanga)
#### Post date: [May 4, 2018, 3:34am UTC](https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389/5 "2018-05-04T03:34:54Z")
@JKhondhu, Thanks. I'll try it.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [June 1, 2018, 3:35am UTC](https://discuss.elastic.co/t/cluster-health-red-unassigned-shards-from-cluster-recovered/129389/6 "2018-06-01T03:35:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.