# Shared File System Gateway behavior **URL:** \ **Category:** Elasticsearch\ **Created:** [May 18, 2012, 6:40am UTC](https://discuss.elastic.co/t/shared-file-system-gateway-behavior/7757 "2012-05-18T06:40:44Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![sujoysett](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sujoysett](https://discuss.elastic.co/u/sujoysett)\ **Post date:** [May 18, 2012, 6:40am UTC](https://discuss.elastic.co/t/shared-file-system-gateway-behavior/7757/1 "2012-05-18T06:40:44Z")
Hi, I am indexing documents from social media into an ES index spanned over two nodes (5 shards 1 replica) on two physical machines, and I am using a third machine as location for the Shared FileSystem Gateway. What happens when this third machine hosting the gateway gets corrupted for any reason? Let us assume a situation, all three machines are restarted, and somehow the third machine hosting the gateway is not starting. In such case, ES fails to start stating that the Gateway Location is unavailable. If we define some other location (empty folder) as gateway, ES starts, but it does not find the metadata of indexes, shows a message that local indexes are dangling indexes, and schedules to delete them from the local store. In other words, if a shared gateway location is permanently lost, what is the procedure to make a copy again from local store to a new gateway location? Regards, Sujoy. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 20, 2012, 8:18pm UTC](https://discuss.elastic.co/t/shared-file-system-gateway-behavior/7757/2 "2012-05-20T20:18:44Z")
If the shared gateway data is lost, then you can't recover from it. Use local gateway, it is the preferred solution. On Fri, May 18, 2012 at 8:40 AM, Sujoy Sett [sujoysett@gmail.com](mailto:sujoysett@gmail.com) wrote: > Hi, > > I am indexing documents from social media into an ES index spanned over > two nodes (5 shards 1 replica) on two physical machines, and I am using a > third machine as location for the Shared FileSystem Gateway. What happens > when this third machine hosting the gateway gets corrupted for any reason? > Let us assume a situation, all three machines are restarted, and somehow > the third machine hosting the gateway is not starting. > > In such case, ES fails to start stating that the Gateway Location is > unavailable. If we define some other location (empty folder) as gateway, ES > starts, but it does not find the metadata of indexes, shows a message that > local indexes are dangling indexes, and schedules to delete them from the > local store. > > In other words, if a shared gateway location is permanently lost, what is > the procedure to make a copy again from local store to a new gateway > location? > > Regards, > Sujoy. ---
**Author:** ![sujoysett](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sujoysett](https://discuss.elastic.co/u/sujoysett)\ **Post date:** [May 21, 2012, 7:31am UTC](https://discuss.elastic.co/t/shared-file-system-gateway-behavior/7757/3 "2012-05-21T07:31:38Z")
Thanks Shay. I will consider moving back to local gateway in that case. But is there any chance of getting API control of moving data from local copy to gateway, instead of the normal flow, if required, for such cases as I mentioned above, in future? And any suggestion on backup strategy? This shared gateway was being used as single point of weekly disk backup for a cluster of two nodes. If I move to local gateway, backup would have to be done in the local servers separately, which becomes increasingly complex when more nodes are involved. Regards, Sujoy. On Monday, May 21, 2012 1:48:44 AM UTC+5:30, kimchy wrote: > If the shared gateway data is lost, then you can't recover from it. Use > local gateway, it is the preferred solution. > > On Fri, May 18, 2012 at 8:40 AM, Sujoy Sett wrote: > > > Hi, > > > > I am indexing documents from social media into an ES index spanned over > > two nodes (5 shards 1 replica) on two physical machines, and I am using a > > third machine as location for the Shared FileSystem Gateway. What happens > > when this third machine hosting the gateway gets corrupted for any reason? > > Let us assume a situation, all three machines are restarted, and somehow > > the third machine hosting the gateway is not starting. > > > > In such case, ES fails to start stating that the Gateway Location is > > unavailable. If we define some other location (empty folder) as gateway, ES > > starts, but it does not find the metadata of indexes, shows a message that > > local indexes are dangling indexes, and schedules to delete them from the > > local store. > > > > In other words, if a shared gateway location is permanently lost, what is > > the procedure to make a copy again from local store to a new gateway > > location? > > > > Regards, > > Sujoy. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 23, 2012, 9:08pm UTC](https://discuss.elastic.co/t/shared-file-system-gateway-behavior/7757/4 "2012-05-23T21:08:58Z")
Hi, Yes, one thing that is missing is a backup/restore API when using the local gateway. For now, you can simply backup the data location of each node. On Mon, May 21, 2012 at 9:31 AM, Sujoy Sett [sujoysett@gmail.com](mailto:sujoysett@gmail.com) wrote: > Thanks Shay. > > I will consider moving back to local gateway in that case. > > But is there any chance of getting API control of moving data from local > copy to gateway, instead of the normal flow, if required, for such cases as > I mentioned above, in future? > > And any suggestion on backup strategy? This shared gateway was being used > as single point of weekly disk backup for a cluster of two nodes. If I move > to local gateway, backup would have to be done in the local servers > separately, which becomes increasingly complex when more nodes are involved. > > Regards, > Sujoy. > > On Monday, May 21, 2012 1:48:44 AM UTC+5:30, kimchy wrote: > > > If the shared gateway data is lost, then you can't recover from it. Use > > local gateway, it is the preferred solution. > > > > On Fri, May 18, 2012 at 8:40 AM, Sujoy Sett wrote: > > > > > Hi, > > > > > > I am indexing documents from social media into an ES index spanned over > > > two nodes (5 shards 1 replica) on two physical machines, and I am using a > > > third machine as location for the Shared FileSystem Gateway. What happens > > > when this third machine hosting the gateway gets corrupted for any reason? > > > Let us assume a situation, all three machines are restarted, and somehow > > > the third machine hosting the gateway is not starting. > > > > > > In such case, ES fails to start stating that the Gateway Location is > > > unavailable. If we define some other location (empty folder) as gateway, ES > > > starts, but it does not find the metadata of indexes, shows a message that > > > local indexes are dangling indexes, and schedules to delete them from the > > > local store. > > > > > > In other words, if a shared gateway location is permanently lost, what > > > is the procedure to make a copy again from local store to a new gateway > > > location? > > > > > > Regards, > > > Sujoy. ---
**Author:** ![sujoysett](https://avatars.discourse-cdn.com/v4/letter/s/2acd7d/32.png) [@sujoysett](https://discuss.elastic.co/u/sujoysett)\ **Post date:** [May 24, 2012, 6:09am UTC](https://discuss.elastic.co/t/shared-file-system-gateway-behavior/7757/5 "2012-05-24T06:09:44Z")
Thanks Shay. We will be doing that. Regards, Sujoy. On Thursday, May 24, 2012 2:38:58 AM UTC+5:30, kimchy wrote: > Hi, > > Yes, one thing that is missing is a backup/restore API when using the > local gateway. For now, you can simply backup the data location of each > node. > > On Mon, May 21, 2012 at 9:31 AM, Sujoy Sett wrote: > > > Thanks Shay. > > > > I will consider moving back to local gateway in that case. > > > > But is there any chance of getting API control of moving data from local > > copy to gateway, instead of the normal flow, if required, for such cases as > > I mentioned above, in future? > > > > And any suggestion on backup strategy? This shared gateway was being used > > as single point of weekly disk backup for a cluster of two nodes. If I move > > to local gateway, backup would have to be done in the local servers > > separately, which becomes increasingly complex when more nodes are involved. > > > > Regards, > > Sujoy. > > > > On Monday, May 21, 2012 1:48:44 AM UTC+5:30, kimchy wrote: > > > > > If the shared gateway data is lost, then you can't recover from it. Use > > > local gateway, it is the preferred solution. > > > > > > On Fri, May 18, 2012 at 8:40 AM, Sujoy Sett wrote: > > > > > > > Hi, > > > > > > > > I am indexing documents from social media into an ES index spanned over > > > > two nodes (5 shards 1 replica) on two physical machines, and I am using a > > > > third machine as location for the Shared FileSystem Gateway. What happens > > > > when this third machine hosting the gateway gets corrupted for any reason? > > > > Let us assume a situation, all three machines are restarted, and somehow > > > > the third machine hosting the gateway is not starting. > > > > > > > > In such case, ES fails to start stating that the Gateway Location is > > > > unavailable. If we define some other location (empty folder) as gateway, ES > > > > starts, but it does not find the metadata of indexes, shows a message that > > > > local indexes are dangling indexes, and schedules to delete them from the > > > > local store. > > > > > > > > In other words, if a shared gateway location is permanently lost, what > > > > is the procedure to make a copy again from local store to a new gateway > > > > location? > > > > > > > > Regards, > > > > Sujoy. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:27am UTC](https://discuss.elastic.co/t/shared-file-system-gateway-behavior/7757/6 "2017-07-06T03:27:20Z")