Cloud Plugin, Gateway, and Container (S3 bucket)


(Shay Banon) #1

Hi,

The current cloud plugin creates several containers. One for the cluster
metadata, and 2 for each shard. This was done because of a limit that no
longer exists in S3 regarding the number of files per bucket. Because there
is no limit anymore, and there is still a limit of the number of buckets
created, I just pushed a change that stores everything under the same
container. A specific index can still be configured to be stored on a
different container.

As a side note, the limit on file size (S3 is 5G) is handled by
elasticsearch, since it will break files into chunks of 4G automatically.

Sadly, with this change, a full reindex will be required and creation of a
new container. I will release 0.7.2 soon with this change.

cheers,
shay.banon


(Clinton Gormley) #2

Sadly, with this change, a full reindex will be required and
creation of a new container. I will release 0.7.2 soon with this
change.

A full reindex is required for all users? Or just those using S3?

ta

clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.


(Shay Banon) #3

Just S3 / Cloud gateway.

On Wed, May 19, 2010 at 3:35 PM, Clinton Gormley clinton@iannounce.co.ukwrote:

Sadly, with this change, a full reindex will be required and
creation of a new container. I will release 0.7.2 soon with this
change.

A full reindex is required for all users? Or just those using S3?

ta

clint

--
Web Announcements Limited is a company registered in England and Wales,
with company number 05608868, with registered address at 10 Arvon Road,
London, N5 1PR.


(system) #4