# Why do I loose data?
**URL:** https://discuss.elastic.co/t/why-do-i-loose-data/62707
**Category:** Elasticsearch
**Created:** [October 11, 2016, 12:27pm UTC](https://discuss.elastic.co/t/why-do-i-loose-data/62707 "2016-10-11T12:27:12Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Attila\_Nagy](https://discuss.elastic.co/u/Attila_Nagy)
#### Post date: [October 11, 2016, 12:27pm UTC](https://discuss.elastic.co/t/why-do-i-loose-data/62707/1 "2016-10-11T12:27:12Z")
I have set up a cluster from 20 nodes with several indexes and shards. Each shard has one or two replicas set.
The cluster is quiet, in the means that it doesn't receive any queries.
In this state, I start to kill nodes with a random timeout and immediately restart them. After some minutes, I stop the kills and wait for all nodes to come up.
Some shards remain in "red" state and the cluster health says I have many unassigned shards.
I have some questions:
- what can I do in this case to regain the lost shards?
- what can I do to not loose shards?
\_cluster/reroute?explain returns this for an unavailable shard:
"messages\_12" : {
"shards" : {
"0" : [
{
"state" : "UNASSIGNED",
"primary" : true,
"node" : null,
"relocating\_node" : null,
"shard" : 0,
"index" : "messages\_12",
"recovery\_source" : {
"type" : "EXISTING\_STORE"
},
"unassigned\_info" : {
"reason" : "CLUSTER\_RECOVERED",
"at" : "2016-10-11T11:28:07.442Z",
"delayed" : false,
"allocation\_status" : "no\_valid\_shard\_copy"
}
},
{
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating\_node" : null,
"shard" : 0,
"index" : "messages\_12",
"recovery\_source" : {
"type" : "PEER"
},
"unassigned\_info" : {
"reason" : "CLUSTER\_RECOVERED",
"at" : "2016-10-11T11:28:07.442Z",
"delayed" : false,
"allocation\_status" : "no\_attempt"
}
},
{
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating\_node" : null,
"shard" : 0,
"index" : "messages\_12",
"recovery\_source" : {
"type" : "PEER"
},
"unassigned\_info" : {
"reason" : "CLUSTER\_RECOVERED",
"at" : "2016-10-11T11:28:07.442Z",
"delayed" : false,
"allocation\_status" : "no\_attempt"
}
}
]
}
---
### Author:  [@ywelsch](https://discuss.elastic.co/u/ywelsch)
#### Post date: [October 11, 2016, 1:07pm UTC](https://discuss.elastic.co/t/why-do-i-loose-data/62707/2 "2016-10-11T13:07:30Z")
This is 5.0.0-beta or rc1, right? Can you please share the output of
```auto
curl -XGET 'http://localhost:9200/_shard_stores?pretty'
```
and
```auto
curl -XGET 'http://localhost:9200/_cluster/state?pretty'
```
so I can have a closer look at the state the cluster ended up in.
Also, have you correctly configured the `discovery.zen.minimum_master_nodes` setting?
---
### Author:  [@Attila\_Nagy](https://discuss.elastic.co/u/Attila_Nagy)
#### Post date: [October 12, 2016, 7:19pm UTC](https://discuss.elastic.co/t/why-do-i-loose-data/62707/3 "2016-10-12T19:19:43Z")
Yes, beta1. And indeed, a split brain could occur due to an error (minimum\_master\_nodes wasn't enough for a quorum, every data node was also a master node because of a wrong configuration entry).
Sorry for the noise, I re-did the test with the correct setting and could not see this behaviour.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 10:12pm UTC](https://discuss.elastic.co/t/why-do-i-loose-data/62707/4 "2017-07-05T22:12:55Z")