# Bad recovery after cluster restart **URL:** \ **Category:** Elasticsearch\ **Created:** [November 22, 2011, 12:16pm UTC](https://discuss.elastic.co/t/bad-recovery-after-cluster-restart/5937 "2011-11-22T12:16:16Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Ludovic](https://avatars.discourse-cdn.com/v4/letter/l/7ab992/32.png) [@Ludovic](https://discuss.elastic.co/u/Ludovic)\ **Post date:** [November 22, 2011, 12:16pm UTC](https://discuss.elastic.co/t/bad-recovery-after-cluster-restart/5937/1 "2011-11-22T12:16:16Z")
Hi, We are working on ES 0.17.6 with 2 servers and 4 ES nodes. After a restart on cluster, we have lost all indices. We can see them in work folders but the cluster state is green (cluster health: green (4, 0)). but nothing happens concerning the indices. the gist with configuration and log > any ideas how to force recovery ? Thanks you. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 22, 2011, 12:27pm UTC](https://discuss.elastic.co/t/bad-recovery-after-cluster-restart/5937/2 "2011-11-22T12:27:33Z")
- I am a bit confused about your setup. You are using the shared fs gateway, is the path you configure it with (the work directory of ES???) mounted on both servers and both servers see the same file system? - Why are you using the shared fs gateway and not the local gateway? - Why are you starting two nodes on a single server? On Tue, Nov 22, 2011 at 2:16 PM, Ludovic [superglu07@gmail.com](mailto:superglu07@gmail.com) wrote: > Hi, > > We are working on ES 0.17.6 with 2 servers and 4 ES nodes. > After a restart on cluster, we have lost all indices. > We can see them in work folders but the cluster state is green (cluster > health: green (4, 0)). but nothing happens concerning the indices. > > the gist with configuration and log > > [bad recovery after cluster restart · GitHub](https://gist.github.com/1385395) > > any ideas how to force recovery ? > > Thanks you. ---
**Author:** ![Ludovic](https://avatars.discourse-cdn.com/v4/letter/l/7ab992/32.png) [@Ludovic](https://discuss.elastic.co/u/Ludovic)\ **Post date:** [November 22, 2011, 2:06pm UTC](https://discuss.elastic.co/t/bad-recovery-after-cluster-restart/5937/3 "2011-11-22T14:06:53Z")
Hi Shay - I am a bit confused about your setup. You are using the shared fs gateway, is the path you configure it with (the work directory of ES???) mounted on both servers and both servers see the same file system? each nodes has its own "work" directory. - Why are you using the shared fs gateway and not the local gateway? We have only 2 servers with not enough disk memory, so we can't do anything else then using a disk storage bay. that is why we used this shared fs gateway. - Why are you starting two nodes on a single server? The indices we want to use have 1 replica. We want to test load balancing for 4 nodes and only 2 servers. We understand, but we might be wrong, that 4 nodes will be better than 2. best regards ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 22, 2011, 3:38pm UTC](https://discuss.elastic.co/t/bad-recovery-after-cluster-restart/5937/4 "2011-11-22T15:38:02Z")
On Tue, Nov 22, 2011 at 4:06 PM, Ludovic [superglu07@gmail.com](mailto:superglu07@gmail.com) wrote: > Hi Shay > > - I am a bit confused about your setup. You are using the shared fs > gateway, is the path you configure it with (the work directory of ES???) > mounted on both servers and both servers see the same file system? > > each nodes has its own "work" directory. then this does not work. The shared fs gateway requires the same mount to be visible from all servers. But, you don't really need it, use the default, recommended local gateway. That explains, btw, why you "lost" your data. the master node stores in the presumed shared gateway location the fact that indices were created. But, because your location is not shared, when you started and a master node was elected on another server, it will go to its own "shared" fs location, and there won't be any meta data there. > - Why are you using the shared fs gateway and not the local gateway? > > We have only 2 servers with not enough disk memory, so we can't do > anything else then using a disk storage bay. that is why we used this > shared fs gateway. The local gateway also works using the "file system": [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/modules/gateway/local.html). I suggest you use it. You will need to reindex the data to move from shared fs gateway to local gateway. > - Why are you starting two nodes on a single server? > > The indices we want to use have 1 replica. We want to test load balancing > for 4 nodes and only 2 servers. We understand, but we might be wrong, that > 4 nodes will be better than 2. 99% change you are wrong, use one node per machine. > best regards ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:47am UTC](https://discuss.elastic.co/t/bad-recovery-after-cluster-restart/5937/5 "2017-07-06T03:47:50Z")