# Repeating Error -- No commit point data is available in gateway

**URL:** <https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309>\
**Category:** Elasticsearch\
**Created:** [September 7, 2010, 6:10pm UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309 "2010-09-07T18:10:25Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman/32/2224_2.png) [@Kenneth\_Loafman](https://discuss.elastic.co/u/Kenneth_Loafman)\
**Post date:** [September 7, 2010, 6:10pm UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309/1 "2010-09-07T18:10:25Z")

</div>

Hi,

I'm getting the following block repeating... [http://gist.github.com/568753](http://gist.github.com/568753)

My guess is that no recovery is possible at this point. Is that  
correct, or is there some magic available?

...Thanks,  
...ken

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [September 8, 2010, 9:18am UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309/2 "2010-09-08T09:18:27Z")

</div>

Yea, seems like files got deleted in the gateway storage. Which one type you  
using? In any case, the mentioned index will not be able to recover... .

On Tue, Sep 7, 2010 at 9:10 PM, Kenneth Loafman  
[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)wrote:

> Hi,
> 
> I'm getting the following block repeating... [568753’s gists · GitHub](http://gist.github.com/568753)
> 
> My guess is that no recovery is possible at this point. Is that  
> correct, or is there some magic available?
> 
> ...Thanks,  
> ...ken

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman/32/2224_2.png) [@Kenneth\_Loafman](https://discuss.elastic.co/u/Kenneth_Loafman)\
**Post date:** [September 8, 2010, 12:26pm UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309/3 "2010-09-08T12:26:29Z")

</div>

S3 gateway.

This is the 2nd time this has happened. I'm guessing more along the  
lines that the file never made it to S3 rather than getting deleted  
accidentally.

Anyone have any experience with Rackspace cloud service? Reliable?

Shay Banon wrote:

> Yea, seems like files got deleted in the gateway storage. Which one type  
> you using? In any case, the mentioned index will not be able to recover... .
> 
> On Tue, Sep 7, 2010 at 9:10 PM, Kenneth Loafman  
> \<[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com) [mailto:kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> 
> ```
> Hi,
> 
> I'm getting the following block repeating...
> http://gist.github.com/568753
> 
> My guess is that no recovery is possible at this point. Is that
> correct, or is there some magic available?
> 
> ...Thanks,
> ...ken
> 
> ```

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [September 8, 2010, 7:51pm UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309/4 "2010-09-08T19:51:27Z")

</div>

The problem might be in several different places. The first is  
elasticsearch, it might have either not written the file, or deleted it by  
mistake. Commit points are only written after all the files within it are  
written, and files are deleted once there is not a single commit point  
pointing to it. I can't recreate what you are getting, so its hard for me to  
track down a possible problem in elasticsearch.

The other place the problem might be is the amazon SDK, that for some reason  
does not upload the files correctly, or does not list the resulting files  
correctly. Not sure... .

Last, the objects might have been lost on the s3. I would say (hope) that is  
the most unlikely case.

I will continue to try and track down this problem and report back.

Regarding rackspace and cloudfiles, I am not sure if its better or not. In  
any case, I need to write support for cloudfiles, and auto disco for  
cloudservers. This is on my roadmap for a cloud-rackspace plugin.

-shay.banon

On Wed, Sep 8, 2010 at 3:26 PM, Kenneth Loafman  
[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)wrote:

> S3 gateway.
> 
> This is the 2nd time this has happened. I'm guessing more along the  
> lines that the file never made it to S3 rather than getting deleted  
> accidentally.
> 
> Anyone have any experience with Rackspace cloud service? Reliable?
> 
> Shay Banon wrote:
> 
> > Yea, seems like files got deleted in the gateway storage. Which one type  
> > you using? In any case, the mentioned index will not be able to  
> > recover... .
> > 
> > On Tue, Sep 7, 2010 at 9:10 PM, Kenneth Loafman  
> > \<[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com) [mailto:kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > 
> > ```
> > Hi,
> > 
> > I'm getting the following block repeating...
> > http://gist.github.com/568753
> > 
> > My guess is that no recovery is possible at this point. Is that
> > correct, or is there some magic available?
> > 
> > ...Thanks,
> > ...ken
> > 
> > ```

---

<div class="post-metadata">

**Author:** ![Kenneth\_Loafman](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenneth_loafman/32/2224_2.png) [@Kenneth\_Loafman](https://discuss.elastic.co/u/Kenneth_Loafman)\
**Post date:** [September 8, 2010, 8:33pm UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309/5 "2010-09-08T20:33:46Z")

</div>

Just a note on S3 files listing... 10k files seems to be the limit for  
their API. Is it possible there were that many files to list?

...Ken

Shay Banon wrote:

> The problem might be in several different places. The first is  
> elasticsearch, it might have either not written the file, or deleted it  
> by mistake. Commit points are only written after all the files within it  
> are written, and files are deleted once there is not a single commit  
> point pointing to it. I can't recreate what you are getting, so its hard  
> for me to track down a possible problem in elasticsearch.
> 
> The other place the problem might be is the amazon SDK, that for some  
> reason does not upload the files correctly, or does not list the  
> resulting files correctly. Not sure... .
> 
> Last, the objects might have been lost on the s3. I would say (hope)  
> that is the most unlikely case.
> 
> I will continue to try and track down this problem and report back.
> 
> Regarding rackspace and cloudfiles, I am not sure if its better or not.  
> In any case, I need to write support for cloudfiles, and auto disco for  
> cloudservers. This is on my roadmap for a cloud-rackspace plugin.
> 
> -shay.banon
> 
> On Wed, Sep 8, 2010 at 3:26 PM, Kenneth Loafman  
> \<[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com) [mailto:kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> 
> ```
> S3 gateway.
> 
> This is the 2nd time this has happened. I'm guessing more along the
> lines that the file never made it to S3 rather than getting deleted
> accidentally.
> 
> Anyone have any experience with Rackspace cloud service? Reliable?
> 
> Shay Banon wrote:
> > Yea, seems like files got deleted in the gateway storage. Which
> one type
> > you using? In any case, the mentioned index will not be able to
> recover... .
> >
> > On Tue, Sep 7, 2010 at 9:10 PM, Kenneth Loafman
> > <kenneth.loafman@gmail.com <mailto:kenneth.loafman@gmail.com>
> <mailto:kenneth.loafman@gmail.com
> <mailto:kenneth.loafman@gmail.com>>> wrote:
> >
> > Hi,
> >
> > I'm getting the following block repeating...
> > http://gist.github.com/568753
> >
> > My guess is that no recovery is possible at this point. Is that
> > correct, or is there some magic available?
> >
> > ...Thanks,
> > ...ken
> >
> >
> 
> ```

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [September 8, 2010, 8:36pm UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309/6 "2010-09-08T20:36:47Z")

</div>

I don't think that there are 10k files, but even if thats the case, when  
elasticsearch lists blobs on s3, the listing process gets repeated with  
proper markers if the result is marked as truncated.

On Wed, Sep 8, 2010 at 11:33 PM, Kenneth Loafman  
[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)wrote:

> Just a note on S3 files listing... 10k files seems to be the limit for  
> their API. Is it possible there were that many files to list?
> 
> ...Ken
> 
> Shay Banon wrote:
> 
> > The problem might be in several different places. The first is  
> > elasticsearch, it might have either not written the file, or deleted it  
> > by mistake. Commit points are only written after all the files within it  
> > are written, and files are deleted once there is not a single commit  
> > point pointing to it. I can't recreate what you are getting, so its hard  
> > for me to track down a possible problem in elasticsearch.
> > 
> > The other place the problem might be is the amazon SDK, that for some  
> > reason does not upload the files correctly, or does not list the  
> > resulting files correctly. Not sure... .
> > 
> > Last, the objects might have been lost on the s3. I would say (hope)  
> > that is the most unlikely case.
> > 
> > I will continue to try and track down this problem and report back.
> > 
> > Regarding rackspace and cloudfiles, I am not sure if its better or not.  
> > In any case, I need to write support for cloudfiles, and auto disco for  
> > cloudservers. This is on my roadmap for a cloud-rackspace plugin.
> > 
> > -shay.banon
> > 
> > On Wed, Sep 8, 2010 at 3:26 PM, Kenneth Loafman  
> > \<[kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com) [mailto:kenneth.loafman@gmail.com](mailto:kenneth.loafman@gmail.com)\> wrote:
> > 
> > ```
> > S3 gateway.
> > 
> > This is the 2nd time this has happened. I'm guessing more along the
> > lines that the file never made it to S3 rather than getting deleted
> > accidentally.
> > 
> > Anyone have any experience with Rackspace cloud service? Reliable?
> > 
> > Shay Banon wrote:
> > > Yea, seems like files got deleted in the gateway storage. Which
> > one type
> > > you using? In any case, the mentioned index will not be able to
> > recover... .
> > >
> > > On Tue, Sep 7, 2010 at 9:10 PM, Kenneth Loafman
> > > <kenneth.loafman@gmail.com <mailto:kenneth.loafman@gmail.com>
> > <mailto:kenneth.loafman@gmail.com
> > <mailto:kenneth.loafman@gmail.com>>> wrote:
> > >
> > > Hi,
> > >
> > > I'm getting the following block repeating...
> > > http://gist.github.com/568753
> > >
> > > My guess is that no recovery is possible at this point. Is
> > 
> > ```
> 
> that
> 
> > ```
> > > correct, or is there some magic available?
> > >
> > > ...Thanks,
> > > ...ken
> > >
> > >
> > 
> > ```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:19am UTC](https://discuss.elastic.co/t/repeating-error-no-commit-point-data-is-available-in-gateway/3309/7 "2017-07-06T04:19:34Z")

</div>