# Adding S3 gateway on a local-gateway machine **URL:** \ **Category:** Elasticsearch\ **Created:** [December 23, 2010, 11:12pm UTC](https://discuss.elastic.co/t/adding-s3-gateway-on-a-local-gateway-machine/3698 "2010-12-23T23:12:56Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![mrflip](https://avatars.discourse-cdn.com/v4/letter/m/a9adbd/32.png) [@mrflip](https://discuss.elastic.co/u/mrflip)\ **Post date:** [December 23, 2010, 11:12pm UTC](https://discuss.elastic.co/t/adding-s3-gateway-on-a-local-gateway-machine/3698/1 "2010-12-23T23:12:56Z")
We have a machine set up currently with a local gateway (full config at [https://gist.github.com/f003c19dd0ce53c654cb](https://gist.github.com/f003c19dd0ce53c654cb) ) gateway: type: local index: gateway: snapshot\_interval: -1 snapshot\_on\_close: false We're considering moving the cluster to use the S3 gateway. It's a 16- machine cluster; when all is done it will hold about 11 indexes, 176 shards x 2 (replicas = 1), each of about 5-15GB actual-on-disk usage. Can we switch the cluster over to use the s3 gateway without losing files? I know I'll have to trigger a snapshot using eg. curl -XPOST '[http://localhost:9200/\_gateway/snapshot](http://localhost:9200/_gateway/snapshot)' My concern is that once I update the config, I'll have to restart each data node; will it try to initiate recovery from the (empty) s3 gateway, or can I make it adopt the local files already presence and then push them to S3 after going green? Also, are there any non-obvious performance implications for pushing that much data through s3? Will new nodes recover from their peers or pull from s3? thanks, flip ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 23, 2010, 11:26pm UTC](https://discuss.elastic.co/t/adding-s3-gateway-on-a-local-gateway-machine/3698/2 "2010-12-23T23:26:06Z")
Hi, There is no way to switch from local to gateway without reindexing the data. Regarding the overhead of s3, there are basically two. The first is the initial recovery on full cluster startup. If you set the gateway.recovery\_after\_xxx settings, then shards will be allocated to nodes that have the most common local data with regards to s3, so the recovery times should be minimal. The second problem with s3 is more concerning, which is the need to push the data to s3. This will require network resources, which are very rare on ec2 ;), and will compete with indexing / searching network operations... . -shay.banon On Fri, Dec 24, 2010 at 1:12 AM, mrflip [mrflip@gmail.com](mailto:mrflip@gmail.com) wrote: > We have a machine set up currently with a local gateway (full config > at [elasticsearch.yml ยท GitHub](https://gist.github.com/f003c19dd0ce53c654cb) ) > > gateway: > type: local > index: > gateway: > snapshot\_interval: -1 > snapshot\_on\_close: false > > We're considering moving the cluster to use the S3 gateway. It's a 16- > machine cluster; when all is done it will hold about 11 indexes, 176 > shards x 2 (replicas = 1), each of about 5-15GB actual-on-disk usage. > > Can we switch the cluster over to use the s3 gateway without losing > files? > I know I'll have to trigger a snapshot using eg. > curl -XPOST '[http://localhost:9200/\_gateway/snapshot](http://localhost:9200/_gateway/snapshot)' > My concern is that once I update the config, I'll have to restart each > data node; will it try to initiate recovery from the (empty) s3 > gateway, or can I make it adopt the local files already presence and > then push them to S3 after going green? > > Also, are there any non-obvious performance implications for pushing > that much data through s3? Will new nodes recover from their peers or > pull from s3? > > thanks, > flip ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:14am UTC](https://discuss.elastic.co/t/adding-s3-gateway-on-a-local-gateway-machine/3698/3 "2017-07-06T04:14:50Z")