# Cloud Plugin, Gateway, and Container (S3 bucket)

**URL:** <https://discuss.elastic.co/t/cloud-plugin-gateway-and-container-s3-bucket/2958>\
**Category:** Elasticsearch\
**Created:** [May 18, 2010, 8:42pm UTC](https://discuss.elastic.co/t/cloud-plugin-gateway-and-container-s3-bucket/2958 "2010-05-18T20:42:16Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [May 18, 2010, 8:42pm UTC](https://discuss.elastic.co/t/cloud-plugin-gateway-and-container-s3-bucket/2958/1 "2010-05-18T20:42:16Z")

</div>

Hi,

The current cloud plugin creates several containers. One for the cluster  
metadata, and 2 for each shard. This was done because of a limit that no  
longer exists in S3 regarding the number of files per bucket. Because there  
is no limit anymore, and there is still a limit of the number of buckets  
created, I just pushed a change that stores everything under the same  
container. A specific index can still be configured to be stored on a  
different container.

As a side note, the limit on file size (S3 is 5G) is handled by  
elasticsearch, since it will break files into chunks of 4G automatically.

Sadly, with this change, a full reindex will be required and creation of a  
new container. I will release 0.7.2 soon with this change.

cheers,  
shay.banon

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [May 19, 2010, 12:35pm UTC](https://discuss.elastic.co/t/cloud-plugin-gateway-and-container-s3-bucket/2958/2 "2010-05-19T12:35:38Z")

</div>

> Sadly, with this change, a full reindex will be required and  
> creation of a new container. I will release 0.7.2 soon with this  
> change.

A full reindex is required for all users? Or just those using S3?

ta

clint

--  
Web Announcements Limited is a company registered in England and Wales,  
with company number 05608868, with registered address at 10 Arvon Road,  
London, N5 1PR.

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [May 19, 2010, 4:15pm UTC](https://discuss.elastic.co/t/cloud-plugin-gateway-and-container-s3-bucket/2958/3 "2010-05-19T16:15:18Z")

</div>

Just S3 / Cloud gateway.

On Wed, May 19, 2010 at 3:35 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote:

> > Sadly, with this change, a full reindex will be required and  
> > creation of a new container. I will release 0.7.2 soon with this  
> > change.
> 
> A full reindex is required for all users? Or just those using S3?
> 
> ta
> 
> clint
> 
> --  
> Web Announcements Limited is a company registered in England and Wales,  
> with company number 05608868, with registered address at 10 Arvon Road,  
> London, N5 1PR.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:23am UTC](https://discuss.elastic.co/t/cloud-plugin-gateway-and-container-s3-bucket/2958/4 "2017-07-06T04:23:57Z")

</div>

Closed on July 6, 2017, 4:23am UTC