# Is it possible to create a hidden elasticsearch node for urgent recovery?

**URL:** <https://discuss.elastic.co/t/is-it-possible-to-create-a-hidden-elasticsearch-node-for-urgent-recovery/12486>\
**Category:** Elasticsearch\
**Created:** [June 20, 2013, 3:58am UTC](https://discuss.elastic.co/t/is-it-possible-to-create-a-hidden-elasticsearch-node-for-urgent-recovery/12486 "2013-06-20T03:58:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![deer](https://avatars.discourse-cdn.com/v4/letter/d/f19dbf/32.png) [@deer](https://discuss.elastic.co/u/deer)\
**Post date:** [June 20, 2013, 3:58am UTC](https://discuss.elastic.co/t/is-it-possible-to-create-a-hidden-elasticsearch-node-for-urgent-recovery/12486/1 "2013-06-20T03:58:00Z")

</div>

Hi All,

Last time, our elasticsearch(version 0.19) cannot work properly for both  
searching and indexing.  
It is because of the OOM happens in one of the elasticsearch node which  
then make all other nodes cannot work properly after a long time of GC and  
recovery.  
Unfortunately, the replication recovery does not work after restarting all  
nodes. We finally need to re-build the entire indexes.

For the mentioned case, we would like to make the elasticsearch come back  
and work properly in short time(not to re-build the entrie indexes because  
the re-indexing takes about one day).

We have upgraded to 0.90.1 and hope we can avoid the mentioned case. Also  
we are looking for more recovery methods to play safe.  
After studying the backup & recovery solutions of other users, we try to  
backup a snapshot and use it when the elasticsearch nodes cannot serve the  
client properly.  
It is simply disable the flush and backup the all indexes under all nodes  
periodically.  
But the data files are too large. Also, backup of all nodes take long time.  
We cannot do the copy and delete frequently(data is not up-to-date).

I have an idea for urgent recovery in our case.  
It is to create a hidden cluster which has the same config with the public  
cluster. We do the same index updates to both public cluster and the hidden  
cluster.  
When there is a need of urgent recovery, we copy the entire data from  
hidden cluster to public cluster and restart the public cluster.  
Also, the hidden cluster does not serve the client and it is just for index  
updates. the hidden cluster should be more stable.

May I know does it make sense for me to do this? or, can we make a hidden  
node(has all primary shards) like this?

Thanks.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:30am UTC](https://discuss.elastic.co/t/is-it-possible-to-create-a-hidden-elasticsearch-node-for-urgent-recovery/12486/2 "2017-07-06T02:30:18Z")

</div>

Closed on July 6, 2017, 2:30am UTC