# Recover after failure (lost shards) **URL:** \ **Category:** Elasticsearch\ **Created:** [May 7, 2013, 8:33pm UTC](https://discuss.elastic.co/t/recover-after-failure-lost-shards/11864 "2013-05-07T20:33:57Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [May 7, 2013, 8:33pm UTC](https://discuss.elastic.co/t/recover-after-failure-lost-shards/11864/1 "2013-05-07T20:33:57Z")
Heya, I just ran into a major node HDD failure and lost some shards on as index, so this is bad, but not that bad since we are in the process or reindexing everything. However, I would like to replace the lost shards with "dummy" empty shards just to retrieve the green cluster state. I've read somewhere ( [http://elasticsearch-users.115913.n3.nabble.com/Recovering-after-shard-failure-td4018776.html](http://elasticsearch-users.115913.n3.nabble.com/Recovering-after-shard-failure-td4018776.html)) that creating a similar index on a local cluster and moving the empty missing shards to some of the cluster nodes and they will be recovered. However, when I tried it, I got this error; > I was wondering, is this method still valid with 0.20.x versions ? And if not, is there a way to do it ? I would be nice to have this... I've seen some post about it, but the only solution I have found is the one I mentionned. Many thanks, Jerome -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [May 7, 2013, 8:44pm UTC](https://discuss.elastic.co/t/recover-after-failure-lost-shards/11864/2 "2013-05-07T20:44:00Z")
Oh, forgot to mention; we don't have replicas on this index. (We are currently adding them, while reindexing) Don't do that by the way (no replicas), its a massive pain. 馃檪 On Tuesday, May 7, 2013 4:33:57 PM UTC-4, J茅r么me Gagnon wrote: > Heya, > > I just ran into a major node HDD failure and lost some shards on as index, > so this is bad, but not that bad since we are in the process or reindexing > everything. However, I would like to replace the lost shards with "dummy" > empty shards just to retrieve the green cluster state. > I've read somewhere ( > [http://elasticsearch-users.115913.n3.nabble.com/Recovering-after-shard-failure-td4018776.html](http://elasticsearch-users.115913.n3.nabble.com/Recovering-after-shard-failure-td4018776.html)) > that creating a similar index on a local cluster and moving the empty > missing shards to some of the cluster nodes and they will be recovered. > > However, when I tried it, I got this error; > [gist:a4ebc27b17e8451af59e 路 GitHub](https://gist.github.com/jgagnon1/a4ebc27b17e8451af59e) > > I was wondering, is this method still valid with 0.20.x versions ? And if > not, is there a way to do it ? I would be nice to have this... I've seen > some post about it, but the only solution I have found is the one I > mentionned. > > Many thanks, > > Jerome -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [May 8, 2013, 1:37pm UTC](https://discuss.elastic.co/t/recover-after-failure-lost-shards/11864/3 "2013-05-08T13:37:01Z")
Thanks to Shay, I finally got it working; Basically the solution was the good one; but I made two mistakes; 1. Shard folder permissions; the shard folder must have the same owner/permission as the other (user that run ES process) 2. Created a "dummy index" from another cluster which was running Elasticsearch 0.90.x which was creating Lucene 4.x shard instead of 3.6 Shay told me that in 0.90.x the error message when ES is not able to load some Lucene shard are much clearer, good to know! But then again, we are running without replicas on one index due to several constraint (IO, etc...), and this is not the way to go, we've been butt hurt (and still are) many time due to that, and we are moving forward with a better architecture. On the end, I got the cluster green and learned more. and finally went to sleep, everything's good 馃檪 Jerome On Tuesday, May 7, 2013 4:44:00 PM UTC-4, J茅r么me Gagnon wrote: > Oh, forgot to mention; we don't have replicas on this index. (We are > currently adding them, while reindexing) > > Don't do that by the way (no replicas), its a massive pain. 馃檪 > > On Tuesday, May 7, 2013 4:33:57 PM UTC-4, J茅r么me Gagnon wrote: > > > Heya, > > > > I just ran into a major node HDD failure and lost some shards on as > > index, so this is bad, but not that bad since we are in the process or > > reindexing everything. However, I would like to replace the lost shards > > with "dummy" empty shards just to retrieve the green cluster state. > > I've read somewhere ( > > [http://elasticsearch-users.115913.n3.nabble.com/Recovering-after-shard-failure-td4018776.html](http://elasticsearch-users.115913.n3.nabble.com/Recovering-after-shard-failure-td4018776.html)) > > that creating a similar index on a local cluster and moving the empty > > missing shards to some of the cluster nodes and they will be recovered. > > > > However, when I tried it, I got this error; > > [gist:a4ebc27b17e8451af59e 路 GitHub](https://gist.github.com/jgagnon1/a4ebc27b17e8451af59e) > > > > I was wondering, is this method still valid with 0.20.x versions ? And if > > not, is there a way to do it ? I would be nice to have this... I've seen > > some post about it, but the only solution I have found is the one I > > mentionned. > > > > Many thanks, > > > > Jerome -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:37am UTC](https://discuss.elastic.co/t/recover-after-failure-lost-shards/11864/4 "2017-07-06T02:37:38Z")