# Logging reallocation detail during node restore

**URL:** <https://discuss.elastic.co/t/logging-reallocation-detail-during-node-restore/8813>\
**Category:** Elasticsearch\
**Created:** [August 22, 2012, 1:54pm UTC](https://discuss.elastic.co/t/logging-reallocation-detail-during-node-restore/8813 "2012-08-22T13:54:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![John\_Nader](https://avatars.discourse-cdn.com/v4/letter/j/4bbf92/32.png) [@John\_Nader](https://discuss.elastic.co/u/John_Nader)\
**Post date:** [August 22, 2012, 1:54pm UTC](https://discuss.elastic.co/t/logging-reallocation-detail-during-node-restore/8813/1 "2012-08-22T13:54:56Z")

</div>

We are using ES version 0.19.8 in a 10 node cluster with 30 shards.

We had an incident yesterday where the process of reallocating shards to a  
node simply hung. The head console showed the node came up health, and  
that two shards were in the process of being reallocated to that node.  
However, no data transfer occurred (logging at target directory) and  
logging showed any progress or issues from either the source nodes or the  
new destination node. We were able to recover by bringing the node down  
and back up. We have not see this issue again.

I am looking for advice on logging reallocation information such that we  
can troubleshoot such an issue if it occurs again. Can someone recommend  
settings that will give good detail on reallocation start, progress, and  
completion without logging other excessive detail that fills up our logging  
directories or makes it difficult to find the relevant entries in the log?

Thanks!

-John

--

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [August 22, 2012, 4:21pm UTC](https://discuss.elastic.co/t/logging-reallocation-detail-during-node-restore/8813/2 "2012-08-22T16:21:02Z")

</div>

There isn't a low level recovery logging (i.e. log a message on each chunk of data being recovered) or things of that nature. It probably make sense to add it, just so we can more easily try and resolve something like this. Care to open an issue?

On Aug 22, 2012, at 3:54 PM, John Nader [nadernader99@gmail.com](mailto:nadernader99@gmail.com) wrote:

> We are using ES version 0.19.8 in a 10 node cluster with 30 shards.
> 
> We had an incident yesterday where the process of reallocating shards to a node simply hung. The head console showed the node came up health, and that two shards were in the process of being reallocated to that node. However, no data transfer occurred (logging at target directory) and logging showed any progress or issues from either the source nodes or the new destination node. We were able to recover by bringing the node down and back up. We have not see this issue again.
> 
> I am looking for advice on logging reallocation information such that we can troubleshoot such an issue if it occurs again. Can someone recommend settings that will give good detail on reallocation start, progress, and completion without logging other excessive detail that fills up our logging directories or makes it difficult to find the relevant entries in the log?
> 
> Thanks!
> 
> -John
> 
> --

--

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:15am UTC](https://discuss.elastic.co/t/logging-reallocation-detail-during-node-restore/8813/3 "2017-07-06T03:15:29Z")

</div>


