# Failed recovering a shard after cluster restart
**URL:** https://discuss.elastic.co/t/failed-recovering-a-shard-after-cluster-restart/12384
**Category:** Elasticsearch
**Created:** [June 12, 2013, 3:03pm UTC](https://discuss.elastic.co/t/failed-recovering-a-shard-after-cluster-restart/12384 "2013-06-12T15:03:06Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@aric\_lasry](https://discuss.elastic.co/u/aric_lasry)
#### Post date: [June 12, 2013, 3:03pm UTC](https://discuss.elastic.co/t/failed-recovering-a-shard-after-cluster-restart/12384/1 "2013-06-12T15:03:06Z")
Hi,
I have an elasticSearch cluster of 4 nodes. After restarting each one of
them for some configuration changes, I got into the "red" state. I didn't
really understand what was happening and I restarted the cluster again.
After few minutes, I got back to "green" state but the number of documents
has dropped from ~10 000 000 to ~8 000 000. I tried to refresh the index or
to run a "\_count" query, but I'm still getting the same result.
I looked into the logs and apparently, there is an issue with the
recovering of one shard.
Here are the detailed logs
: [https://gist.github.com/lasryaric/7369e6c1cc3b9619457c](https://gist.github.com/lasryaric/7369e6c1cc3b9619457c)
\_cluster/health: [https://gist.github.com/lasryaric/c248b7b3b381d4394f77](https://gist.github.com/lasryaric/c248b7b3b381d4394f77)
\_nodes?all : [https://gist.github.com/lasryaric/afb25863ed49fdfc864d](https://gist.github.com/lasryaric/afb25863ed49fdfc864d)
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [June 16, 2013, 5:43pm UTC](https://discuss.elastic.co/t/failed-recovering-a-shard-after-cluster-restart/12384/2 "2013-06-16T17:43:59Z")
Hey
you can out of memory on that node (from then on its behaviour is basically
undefined). Do your nodes have a different setup? Just wondering why it did
not happen on other nodes...
On Wed, Jun 12, 2013 at 5:03 PM, aric lasry [lasry.aric@gmail.com](mailto:lasry.aric@gmail.com) wrote:
> Hi,
>
> I have an elasticSearch cluster of 4 nodes. After restarting each one of
> them for some configuration changes, I got into the "red" state. I didn't
> really understand what was happening and I restarted the cluster again.
> After few minutes, I got back to "green" state but the number of documents
> has dropped from ~10 000 000 to ~8 000 000. I tried to refresh the index or
> to run a "\_count" query, but I'm still getting the same result.
>
> I looked into the logs and apparently, there is an issue with the
> recovering of one shard.
>
> Here are the detailed logs :
> [es\_failed\_shard.log · GitHub](https://gist.github.com/lasryaric/7369e6c1cc3b9619457c)
> \_cluster/health: [host:9200/\_cluster/health · GitHub](https://gist.github.com/lasryaric/c248b7b3b381d4394f77)
> \_nodes?all : [host:9200/\_nodes?all · GitHub](https://gist.github.com/lasryaric/afb25863ed49fdfc864d)
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 2:30am UTC](https://discuss.elastic.co/t/failed-recovering-a-shard-after-cluster-restart/12384/3 "2017-07-06T02:30:58Z")