# No commit point data is available in gateway **URL:** \ **Category:** Elasticsearch\ **Created:** [February 8, 2012, 10:11pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648 "2012-02-08T22:11:54Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![VegHead](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/veghead/32/2411_2.png) [@VegHead](https://discuss.elastic.co/u/VegHead)\ **Post date:** [February 8, 2012, 10:11pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/1 "2012-02-08T22:11:54Z")
We suddenly started seeing this message after launching another instance: ---snip--- [2012-02-08 16:07:42,873][WARN][cluster.action.shard] [es1.dev.example.ec2] sending failed shard for [ideas][3], node[\_lKO3A9mS5W3wrDdvuAZlg], [P], s[INITIALIZING], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[ideas][3] No commit point data is available in gateway]]] [2012-02-08 16:07:43,311][WARN][index.gateway.s3] [es1.dev.studyblue.ec2] [ideas][3] listed commit\_point [commit-16qe]/ [55382], but not all files exists, ignoring [2012-02-08 16:07:43,311][WARN][indices.cluster] [es1.dev.example.ec2] [ideas][3] failed to start shard org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [ideas][3] No commit point data is available in gateway at org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.recover(BlobStoreIndexShardGateway.java: 434) at org.elasticsearch.index.gateway.IndexShardGatewayService $1.run(IndexShardGatewayService.java:179) at java.util.concurrent.ThreadPoolExecutor $Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor $Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) ---snip--- Everything I've read indicates that the data in shard 3 is lost. While that's really unfortunate, it's not the end of the world. The real problem is that now that all the nodes in the cluster log this message 2-5 times PER SECOND, which is causing our logfiles to fill up. Is there any way to recover from this error? Any way we just tell the cluster to give up and "scrap" the data in shard 3? -S ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 8, 2012, 10:34pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/2 "2012-02-08T22:34:11Z")
You will need to delete the index. Which version are you using? On Thursday, February 9, 2012 at 12:11 AM, VegHead wrote: > We suddenly started seeing this message after launching another > instance: > > ---snip--- > [2012-02-08 16:07:42,873][WARN][cluster.action.shard] > [es1.dev.example.ec2] sending failed shard for [ideas][3], > node[\_lKO3A9mS5W3wrDdvuAZlg], [P], s[INITIALIZING], reason [Failed to > start shard, message [IndexShardGatewayRecoveryException[[ideas][3] No > commit point data is available in gateway]]] > [2012-02-08 16:07:43,311][WARN][index.gateway.s3] > [es1.dev.studyblue.ec2] [ideas][3] listed commit\_point [commit-16qe]/ > [55382], but not all files exists, ignoring > [2012-02-08 16:07:43,311][WARN][indices.cluster] > [es1.dev.example.ec2] [ideas][3] failed to start shard > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > [ideas][3] No commit point data is available in gateway > at > org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.recover(BlobStoreIndexShardGateway.java: > 434) > at org.elasticsearch.index.gateway.IndexShardGatewayService > $1.run(IndexShardGatewayService.java:179) > at java.util.concurrent.ThreadPoolExecutor > $Worker.runTask(ThreadPoolExecutor.java:886) > at java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:662) > ---snip--- > > Everything I've read indicates that the data in shard 3 is lost. While > that's really unfortunate, it's not the end of the world. The real > problem is that now that all the nodes in the cluster log this message > 2-5 times PER SECOND, which is causing our logfiles to fill up. > > Is there any way to recover from this error? Any way we just tell the > cluster to give up and "scrap" the data in shard 3? > > -S ---
**Author:** ![VegHead](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/veghead/32/2411_2.png) [@VegHead](https://discuss.elastic.co/u/VegHead)\ **Post date:** [February 9, 2012, 4:19pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/3 "2012-02-09T16:19:28Z")
That's extremely unfortunate... I was really hoping to avoid throwing everything out, since it takes a while to reindex 30 million documents... Two nodes with 0.18.6 and one that I had just upgraded to 0.18.7. -S On Feb 8, 4:34 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > You will need to delete the index. Which version are you using? > > On Thursday, February 9, 2012 at 12:11 AM, VegHead wrote: > > > We suddenly started seeing this message after launching another > > instance: > > > ---snip--- > > [2012-02-08 16:07:42,873][WARN][cluster.action.shard] > > [es1.dev.example.ec2] sending failed shard for [ideas][3], > > node[\_lKO3A9mS5W3wrDdvuAZlg], [P], s[INITIALIZING], reason [Failed to > > start shard, message [IndexShardGatewayRecoveryException[[ideas][3] No > > commit point data is available in gateway]]] > > [2012-02-08 16:07:43,311][WARN][index.gateway.s3] > > [es1.dev.studyblue.ec2] [ideas][3] listed commit\_point [commit-16qe]/ > > [55382], but not all files exists, ignoring > > [2012-02-08 16:07:43,311][WARN][indices.cluster] > > [es1.dev.example.ec2] [ideas][3] failed to start shard > > org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: > > [ideas][3] No commit point data is available in gateway > > at > > org.elasticsearch.index.gateway.blobstore.BlobStoreIndexShardGateway.recove r(BlobStoreIndexShardGateway.java: > > 434) > > at org.elasticsearch.index.gateway.IndexShardGatewayService > > $1.run(IndexShardGatewayService.java:179) > > at java.util.concurrent.ThreadPoolExecutor > > $Worker.runTask(ThreadPoolExecutor.java:886) > > at java.util.concurrent.ThreadPoolExecutor > > $Worker.run(ThreadPoolExecutor.java:908) > > at java.lang.Thread.run(Thread.java:662) > > ---snip--- > > > Everything I've read indicates that the data in shard 3 is lost. While > > that's really unfortunate, it's not the end of the world. The real > > problem is that now that all the nodes in the cluster log this message > > 2-5 times PER SECOND, which is causing our logfiles to fill up. > > > Is there any way to recover from this error? Any way we just tell the > > cluster to give up and "scrap" the data in shard 3? ---
**Author:** ![Paul\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_smith/32/1323_2.png) [@Paul\_Smith](https://discuss.elastic.co/u/Paul_Smith)\ **Post date:** [February 9, 2012, 11:15pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/4 "2012-02-09T23:15:02Z")
On 10 February 2012 03:19, VegHead [organicveggie@gmail.com](mailto:organicveggie@gmail.com) wrote: > That's extremely unfortunate... I was really hoping to avoid throwing > everything out, since it takes a while to reindex 30 million > documents... > > Two nodes with 0.18.6 and one that I had just upgraded to 0.18.7. Shay, is there an API way to delete a specific 'bad' shard of an index? I'm not familiar with this type of problem, and my answer here probably won't help because it presumes a certain pattern of indexing your data but if there was a way to delete the bad shard (if it is a bad shard issue) and you did store the ID & Version field in elasticsearch a value matching whatever primary key ID comes from your source of truth (say, the data you indexed comes from a DB and your Id) then this is exactly what Scrutineer can help with: [https://github.com/Aconex/scrutineer](https://github.com/Aconex/scrutineer) If the bad shard could be deleted, leaving a hole in your index data for sure, then Scutineer can let you work out which Ids are missing and you can reindex just those. Lets say you had 5 shards, and 1 shard went bad, then you only need to reindex 20% of your data. Yes, this is a little plug for Scrutineer, and probably won't help you in this case, but maybe for next time you might be able to use Scrutineer to recover from failure scenarios quicker. Having worked with Lucene related systems for 6 years now all I can say is that planning for the Absolute Worst Case scenario for large indexes is always a good idea. Everyone should be _presuming_ something terrible is going to happen to their index one day (could be as simple as someone accidentally issuing a Delete index! no really! that can happen if someone thinks they're connected to the wrong box...) cheers, Paul Smith ---
**Author:** ![Wes\_Plunk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wes_plunk/32/2446_2.png) [@Wes\_Plunk](https://discuss.elastic.co/u/Wes_Plunk)\ **Post date:** [February 10, 2012, 4:27pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/5 "2012-02-10T16:27:20Z")
Shay, Does this error mean that all replicas are bad as well and the index can't recover the shard from a replica? Besides improving query performance don't replicas also provide failover when the shard itself is offline, corrupt, or otherwise unusable? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 12, 2012, 2:17pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/6 "2012-02-12T14:17:26Z")
Heya Paul, No, there isn't a way to say: ok, I understand this shard is in a bad state, or I will never be able to recover it (because I lost all replicas of it with local gateway, or because the shared gateway state got corrupted in a shared gateway), so go ahead and create a fresh copy of it. We can possibly add it as an API option. On Friday, February 10, 2012 at 1:15 AM, Paul Smith wrote: > On 10 February 2012 03:19, VegHead \<[organicveggie@gmail.com](mailto:organicveggie@gmail.com) ([mailto:organicveggie@gmail.com](mailto:organicveggie@gmail.com))\> wrote: > > > That's extremely unfortunate... I was really hoping to avoid throwing > > everything out, since it takes a while to reindex 30 million > > documents... > > > > Two nodes with 0.18.6 and one that I had just upgraded to 0.18.7. > > Shay, is there an API way to delete a specific 'bad' shard of an index? I'm not familiar with this type of problem, and my answer here probably won't help because it presumes a certain pattern of indexing your data but if there was a way to delete the bad shard (if it is a bad shard issue) and you did store the ID & Version field in elasticsearch a value matching whatever primary key ID comes from your source of truth (say, the data you indexed comes from a DB and your Id) then this is exactly what Scrutineer can help with: > > [https://github.com/Aconex/scrutineer](https://github.com/Aconex/scrutineer) > > If the bad shard could be deleted, leaving a hole in your index data for sure, then Scutineer can let you work out which Ids are missing and you can reindex just those. Lets say you had 5 shards, and 1 shard went bad, then you only need to reindex 20% of your data. > > Yes, this is a little plug for Scrutineer, and probably won't help you in this case, but maybe for next time you might be able to use Scrutineer to recover from failure scenarios quicker. Having worked with Lucene related systems for 6 years now all I can say is that planning for the Absolute Worst Case scenario for large indexes is always a good idea. Everyone should be _presuming_ something terrible is going to happen to their index one day (could be as simple as someone accidentally issuing a Delete index! no really! that can happen if someone thinks they're connected to the wrong box...) > > cheers, > > Paul Smith ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 12, 2012, 2:18pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/7 "2012-02-12T14:18:58Z")
Wes, this only applies in the local gateway case, which is the recommended one to be used. With the shared gateway option (the one used in this case, the s3 one), the shared state (on s3 in this case) is the one that counts, and replicas are only there for search performance, not HA. On Friday, February 10, 2012 at 6:27 PM, Wes Plunk wrote: > Shay, > > Does this error mean that all replicas are bad as well and the index can't recover the shard from a replica? > > Besides improving query performance don't replicas also provide failover when the shard itself is offline, corrupt, or otherwise unusable? ---
**Author:** ![VegHead](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/veghead/32/2411_2.png) [@VegHead](https://discuss.elastic.co/u/VegHead)\ **Post date:** [February 17, 2012, 8:36pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/8 "2012-02-17T20:36:05Z")
I have a dumb question (and I realize it's veering away from the original topic): why do you recommend the local gateway over the S3 gateway? Seems to me that S3, while slower than the local gateway, is far simpler for backups. And keeping the shared state on S3 means less load on other nodes when adding a new node to the cluster. -VegHead On Sunday, February 12, 2012 8:18:58 AM UTC-6, kimchy wrote: > Wes, this only applies in the local gateway case, which is the recommended > one to be used. With the shared gateway option (the one used in this case, > the s3 one), the shared state (on s3 in this case) is the one that counts, > and replicas are only there for search performance, not HA. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 19, 2012, 12:48pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/9 "2012-02-19T12:48:30Z")
The reason I recommend using local gateway is because of the overhead of keeping on copying the data to s3 from the "local" storage of each node. You could reduce the "snapshot" interval and snapshot less often, but then, when doing a full restart, then you will "miss" more, since the system always recovers and syncs against s3. The best solution is a combination. Use local gateway, but be able to backup into s3. Full restart will still use the local gateway, and only if explicitly desired, a restore operation can be done to restore from a specific backup from s3. On Friday, February 17, 2012 at 10:36 PM, VegHead wrote: > I have a dumb question (and I realize it's veering away from the original topic): why do you recommend the local gateway over the S3 gateway? > > Seems to me that S3, while slower than the local gateway, is far simpler for backups. And keeping the shared state on S3 means less load on other nodes when adding a new node to the cluster. > > -VegHead > > On Sunday, February 12, 2012 8:18:58 AM UTC-6, kimchy wrote: > > > Wes, this only applies in the local gateway case, which is the recommended one to be used. With the shared gateway option (the one used in this case, the s3 one), the shared state (on s3 in this case) is the one that counts, and replicas are only there for search performance, not HA. ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [February 20, 2012, 12:52am UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/10 "2012-02-20T00:52:06Z")
Hi Shay, just to make it clear regarding the combination, when using local gateway then to be able to backup into S3 (or other storage) one has to do that manually, ie, there is no direct support in ES for local gateway to allow also S3 "snapshotting" right? And when you say "Full restart will still use the local gateway, and only if explicitly desired, a restore operation can be done to restore from a specific backup from s3." what do you exactly mean? Say I use local gateway and in crash I lose some data (can not recover from local gateway). How can I explicitly tell ES node to use one time recovery from shared storage? Did you mean I need to manually copy node local gateway snapshot to the node data folder before node start? Regards, Lukas On Sun, Feb 19, 2012 at 1:48 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > The reason I recommend using local gateway is because of the overhead of > keeping on copying the data to s3 from the "local" storage of each node. > You could reduce the "snapshot" interval and snapshot less often, but then, > when doing a full restart, then you will "miss" more, since the system > always recovers and syncs against s3. > > The best solution is a combination. Use local gateway, but be able to > backup into s3. Full restart will still use the local gateway, and only if > explicitly desired, a restore operation can be done to restore from a > specific backup from s3. > > On Friday, February 17, 2012 at 10:36 PM, VegHead wrote: > > I have a dumb question (and I realize it's veering away from the original > topic): why do you recommend the local gateway over the S3 gateway? > > Seems to me that S3, while slower than the local gateway, is far simpler > for backups. And keeping the shared state on S3 means less load on other > nodes when adding a new node to the cluster. > > -VegHead > > On Sunday, February 12, 2012 8:18:58 AM UTC-6, kimchy wrote: > > Wes, this only applies in the local gateway case, which is the recommended > one to be used. With the shared gateway option (the one used in this case, > the s3 one), the shared state (on s3 in this case) is the one that counts, > and replicas are only there for search performance, not HA. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 20, 2012, 1:11pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/11 "2012-02-20T13:11:14Z")
On Monday, February 20, 2012 at 2:52 AM, Lukáš Vlček wrote: > Hi Shay, > > just to make it clear regarding the combination, when using local gateway then to be able to backup into S3 (or other storage) one has to do that manually, ie, there is no direct support in ES for local gateway to allow also S3 "snapshotting" right? > Right, at least not yet. > > And when you say "Full restart will still use the local gateway, and only if explicitly desired, a restore operation can be done to restore from a specific backup from s3." what do you exactly mean? Say I use local gateway and in crash I lose some data (can not recover from local gateway). How can I explicitly tell ES node to use one time recovery from shared storage? Did you mean I need to manually copy node local gateway snapshot to the node data folder before node start? > If we have a "backup/restore" option, and you lost some data from the local gateway, you could just issue a "restore" API call for the specific indices. > > Regards, > Lukas > > On Sun, Feb 19, 2012 at 1:48 PM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com) ([mailto:kimchy@gmail.com](mailto:kimchy@gmail.com))\> wrote: > > > The reason I recommend using local gateway is because of the overhead of keeping on copying the data to s3 from the "local" storage of each node. You could reduce the "snapshot" interval and snapshot less often, but then, when doing a full restart, then you will "miss" more, since the system always recovers and syncs against s3. > > > > The best solution is a combination. Use local gateway, but be able to backup into s3. Full restart will still use the local gateway, and only if explicitly desired, a restore operation can be done to restore from a specific backup from s3. > > > > On Friday, February 17, 2012 at 10:36 PM, VegHead wrote: > > > > > I have a dumb question (and I realize it's veering away from the original topic): why do you recommend the local gateway over the S3 gateway? > > > > > > Seems to me that S3, while slower than the local gateway, is far simpler for backups. And keeping the shared state on S3 means less load on other nodes when adding a new node to the cluster. > > > > > > -VegHead > > > > > > On Sunday, February 12, 2012 8:18:58 AM UTC-6, kimchy wrote: > > > > > > > Wes, this only applies in the local gateway case, which is the recommended one to be used. With the shared gateway option (the one used in this case, the s3 one), the shared state (on s3 in this case) is the one that counts, and replicas are only there for search performance, not HA. ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [February 20, 2012, 1:52pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/12 "2012-02-20T13:52:34Z")
Shay, I am confused, you are probably talking about possible future "restore" API, but it is not implemented now, right? So right now, if one want to restore from different location then it is manual work (probably copy to "data" folder of the node before it is started, if I understand correctly). Or am I missing something here? Regards, Lukas On Mon, Feb 20, 2012 at 2:11 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > On Monday, February 20, 2012 at 2:52 AM, Lukáš Vlček wrote: > > Hi Shay, > > just to make it clear regarding the combination, when using local gateway > then to be able to backup into S3 (or other storage) one has to do that > manually, ie, there is no direct support in ES for local gateway to allow > also S3 "snapshotting" right? > > Right, at least not yet. > > And when you say "Full restart will still use the local gateway, and only > if explicitly desired, a restore operation can be done to restore from a > specific backup from s3." what do you exactly mean? Say I use local > gateway and in crash I lose some data (can not recover from local gateway). > How can I explicitly tell ES node to use one time recovery from shared > storage? Did you mean I need to manually copy node local gateway snapshot > to the node data folder before node start? > > If we have a "backup/restore" option, and you lost some data from the > local gateway, you could just issue a "restore" API call for the specific > indices. > > Regards, > Lukas > > On Sun, Feb 19, 2012 at 1:48 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > > The reason I recommend using local gateway is because of the overhead of > keeping on copying the data to s3 from the "local" storage of each node. > You could reduce the "snapshot" interval and snapshot less often, but then, > when doing a full restart, then you will "miss" more, since the system > always recovers and syncs against s3. > > The best solution is a combination. Use local gateway, but be able to > backup into s3. Full restart will still use the local gateway, and only if > explicitly desired, a restore operation can be done to restore from a > specific backup from s3. > > On Friday, February 17, 2012 at 10:36 PM, VegHead wrote: > > I have a dumb question (and I realize it's veering away from the original > topic): why do you recommend the local gateway over the S3 gateway? > > Seems to me that S3, while slower than the local gateway, is far simpler > for backups. And keeping the shared state on S3 means less load on other > nodes when adding a new node to the cluster. > > -VegHead > > On Sunday, February 12, 2012 8:18:58 AM UTC-6, kimchy wrote: > > Wes, this only applies in the local gateway case, which is the recommended > one to be used. With the shared gateway option (the one used in this case, > the s3 one), the shared state (on s3 in this case) is the one that counts, > and replicas are only there for search performance, not HA. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 20, 2012, 2:09pm UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/13 "2012-02-20T14:09:39Z")
Yes, "restore" is a future possible API and functionality. On Monday, February 20, 2012 at 3:52 PM, Lukáš Vlček wrote: > Shay, > I am confused, you are probably talking about possible future "restore" API, but it is not implemented now, right? So right now, if one want to restore from different location then it is manual work (probably copy to "data" folder of the node before it is started, if I understand correctly). Or am I missing something here? > Regards, > Lukas > > On Mon, Feb 20, 2012 at 2:11 PM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com) ([mailto:kimchy@gmail.com](mailto:kimchy@gmail.com))\> wrote: > > > On Monday, February 20, 2012 at 2:52 AM, Lukáš Vlček wrote: > > > > > Hi Shay, > > > > > > just to make it clear regarding the combination, when using local gateway then to be able to backup into S3 (or other storage) one has to do that manually, ie, there is no direct support in ES for local gateway to allow also S3 "snapshotting" right? > > > Right, at least not yet. > > > > > > And when you say "Full restart will still use the local gateway, and only if explicitly desired, a restore operation can be done to restore from a specific backup from s3." what do you exactly mean? Say I use local gateway and in crash I lose some data (can not recover from local gateway). How can I explicitly tell ES node to use one time recovery from shared storage? Did you mean I need to manually copy node local gateway snapshot to the node data folder before node start? > > > If we have a "backup/restore" option, and you lost some data from the local gateway, you could just issue a "restore" API call for the specific indices. > > > > > > Regards, > > > Lukas > > > > > > On Sun, Feb 19, 2012 at 1:48 PM, Shay Banon \<[kimchy@gmail.com](mailto:kimchy@gmail.com) ([mailto:kimchy@gmail.com](mailto:kimchy@gmail.com))\> wrote: > > > > > > > The reason I recommend using local gateway is because of the overhead of keeping on copying the data to s3 from the "local" storage of each node. You could reduce the "snapshot" interval and snapshot less often, but then, when doing a full restart, then you will "miss" more, since the system always recovers and syncs against s3. > > > > > > > > The best solution is a combination. Use local gateway, but be able to backup into s3. Full restart will still use the local gateway, and only if explicitly desired, a restore operation can be done to restore from a specific backup from s3. > > > > > > > > On Friday, February 17, 2012 at 10:36 PM, VegHead wrote: > > > > > > > > > I have a dumb question (and I realize it's veering away from the original topic): why do you recommend the local gateway over the S3 gateway? > > > > > > > > > > Seems to me that S3, while slower than the local gateway, is far simpler for backups. And keeping the shared state on S3 means less load on other nodes when adding a new node to the cluster. > > > > > > > > > > -VegHead > > > > > > > > > > On Sunday, February 12, 2012 8:18:58 AM UTC-6, kimchy wrote: > > > > > > > > > > > Wes, this only applies in the local gateway case, which is the recommended one to be used. With the shared gateway option (the one used in this case, the s3 one), the shared state (on s3 in this case) is the one that counts, and replicas are only there for search performance, not HA. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:38am UTC](https://discuss.elastic.co/t/no-commit-point-data-is-available-in-gateway/6648/14 "2017-07-06T03:38:46Z")