# Refusal to recover after node rebuild

**URL:** <https://discuss.elastic.co/t/refusal-to-recover-after-node-rebuild/20185>\
**Category:** Elasticsearch\
**Created:** [October 10, 2014, 4:36pm UTC](https://discuss.elastic.co/t/refusal-to-recover-after-node-rebuild/20185 "2014-10-10T16:36:23Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Duncan\_Innes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/duncan_innes/32/533_2.png) [@Duncan\_Innes](https://discuss.elastic.co/u/Duncan_Innes)\
**Post date:** [October 10, 2014, 4:36pm UTC](https://discuss.elastic.co/t/refusal-to-recover-after-node-rebuild/20185/1 "2014-10-10T16:36:23Z")

</div>

Hi,

I've got a proof of concept cluster with 5 nodes. Several months rsyslog  
data is in there with 2 replicas per index.

I then decided to rebuilt 2 nodes simultaneously. No problem. Cluster  
reallocated as expected and each of the remaining 3 nodes stored all of the  
indexes and replicas in full. Once the cluster had finished this  
reallocation, I decided to rebuild another 2 nodes simultaneously (without  
waiting for the first 2 to come back). This, after all, would leave 1 node  
storing all of the data.

Unfortunately that's where things start to unravel. The initial 2 nodes  
have come back online and joined the cluster. But not my cluster reports  
that every shard is unassigned and there doesn't seem to be any process  
running to reallocate.

What I don't understand is that the cluster was fully balanced at 3 nodes  
and 2 replicas per index. Does taking a node out in this instance cause a  
problem? My data is still sitting on the node that hasn't been rebuilt,  
but I can't get it to reallocate onto the other nodes.

It's only a proof of concept, so data loss isn't the issue here. It's  
understanding why this happened and figuring out if I did anything  
inherently wrong.

Cheers

Duncan

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8a301f07-1634-4b8c-93c7-5a84f45b534e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8a301f07-1634-4b8c-93c7-5a84f45b534e%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:56am UTC](https://discuss.elastic.co/t/refusal-to-recover-after-node-rebuild/20185/2 "2017-07-06T00:56:59Z")

</div>

Closed on July 6, 2017, 12:56am UTC