# How to add ec2 s3 or other gateway after index is created?
**URL:** https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443
**Category:** Elasticsearch
**Created:** [July 17, 2012, 11:10pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443 "2012-07-17T23:10:34Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)
#### Post date: [July 17, 2012, 11:10pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/1 "2012-07-17T23:10:34Z")
hi,
i have a running ES cluster (of 1 node only) on ec2 with lots of data on
it.. this is without ec2 plugin and any gateway. now i want to configure a
s3 gateway on top of it. but if i follow the steps in the tutorial, it ends
up deleting my index! how do i get around this problem? is it even possible?
i took a copy of the data as a backup and hence i am able to retry multiple
times by restoring the data back..
thanks
ps - i followed the steps in the tutorial with a new cluster and it works
perfectly. also, for my experiments, i am renaming the cluster by simply
renaming the cluster name dir in data dir.
---
### Author:  [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)
#### Post date: [July 18, 2012, 7:21pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/2 "2012-07-18T19:21:33Z")
hi,
i tried different routes and still not able to get it to work. any ideas?
i even tried copying the whole data dir structure on s3 and using the same
name as cluster name and yet didnt work.. ES would empty out the cluster
name dir.
thanks
On Tue, Jul 17, 2012 at 4:10 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> hi,
> i have a running ES cluster (of 1 node only) on ec2 with lots of data on
> it.. this is without ec2 plugin and any gateway. now i want to configure a
> s3 gateway on top of it. but if i follow the steps in the tutorial, it ends
> up deleting my index! how do i get around this problem? is it even possible?
>
> i took a copy of the data as a backup and hence i am able to retry
> multiple times by restoring the data back..
>
> thanks
>
> ps - i followed the steps in the tutorial with a new cluster and it works
> perfectly. also, for my experiments, i am renaming the cluster by simply
> renaming the cluster name dir in data dir.
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [July 18, 2012, 8:41pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/3 "2012-07-18T20:41:19Z")
Its recommended to use the default local gateway even on AWS. No need to
move to s3 gateway really.
On Wed, Jul 18, 2012 at 12:21 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> hi,
> i tried different routes and still not able to get it to work. any ideas?
> i even tried copying the whole data dir structure on s3 and using the same
> name as cluster name and yet didnt work.. ES would empty out the cluster
> name dir.
>
> thanks
>
> On Tue, Jul 17, 2012 at 4:10 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
>
> > hi,
> > i have a running ES cluster (of 1 node only) on ec2 with lots of data on
> > it.. this is without ec2 plugin and any gateway. now i want to configure a
> > s3 gateway on top of it. but if i follow the steps in the tutorial, it ends
> > up deleting my index! how do i get around this problem? is it even possible?
> >
> > i took a copy of the data as a backup and hence i am able to retry
> > multiple times by restoring the data back..
> >
> > thanks
> >
> > ps - i followed the steps in the tutorial with a new cluster and it works
> > perfectly. also, for my experiments, i am renaming the cluster by simply
> > renaming the cluster name dir in data dir.
---
### Author:  [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)
#### Post date: [July 18, 2012, 8:47pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/4 "2012-07-18T20:47:54Z")
just to clarify, local default gateway is what you specify in "path.data"
setting right?
what i want to achieve from s3 is to handle a scenario where my node that
uses local ephemeral disk as local data store and then due to some outage
or crash, the local data is lost/corrupted. i should be easily able to
create another instance and point it to s3 and then it restores all the
data locally on that machine and is able to serve data and index data..
is that clear?
thanks
On Wed, Jul 18, 2012 at 1:41 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> Its recommended to use the default local gateway even on AWS. No need to
> move to s3 gateway really.
>
> On Wed, Jul 18, 2012 at 12:21 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
>
> > hi,
> > i tried different routes and still not able to get it to work. any ideas?
> > i even tried copying the whole data dir structure on s3 and using the
> > same name as cluster name and yet didnt work.. ES would empty out the
> > cluster name dir.
> >
> > thanks
> >
> > On Tue, Jul 17, 2012 at 4:10 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> >
> > > hi,
> > > i have a running ES cluster (of 1 node only) on ec2 with lots of data on
> > > it.. this is without ec2 plugin and any gateway. now i want to configure a
> > > s3 gateway on top of it. but if i follow the steps in the tutorial, it ends
> > > up deleting my index! how do i get around this problem? is it even possible?
> > >
> > > i took a copy of the data as a backup and hence i am able to retry
> > > multiple times by restoring the data back..
> > >
> > > thanks
> > >
> > > ps - i followed the steps in the tutorial with a new cluster and it
> > > works perfectly. also, for my experiments, i am renaming the cluster by
> > > simply renaming the cluster name dir in data dir.
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [July 18, 2012, 8:52pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/5 "2012-07-18T20:52:16Z")
It is clear, but you can also simply store it on EBS, or have more nodes
with more copies of your data.
On Wed, Jul 18, 2012 at 1:47 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> just to clarify, local default gateway is what you specify in "path.data"
> setting right?
>
> what i want to achieve from s3 is to handle a scenario where my node that
> uses local ephemeral disk as local data store and then due to some outage
> or crash, the local data is lost/corrupted. i should be easily able to
> create another instance and point it to s3 and then it restores all the
> data locally on that machine and is able to serve data and index data..
>
> is that clear?
>
> thanks
>
> On Wed, Jul 18, 2012 at 1:41 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
>
> > Its recommended to use the default local gateway even on AWS. No need to
> > move to s3 gateway really.
> >
> > On Wed, Jul 18, 2012 at 12:21 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> >
> > > hi,
> > > i tried different routes and still not able to get it to work. any ideas?
> > > i even tried copying the whole data dir structure on s3 and using the
> > > same name as cluster name and yet didnt work.. ES would empty out the
> > > cluster name dir.
> > >
> > > thanks
> > >
> > > On Tue, Jul 17, 2012 at 4:10 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> > >
> > > > hi,
> > > > i have a running ES cluster (of 1 node only) on ec2 with lots of data
> > > > on it.. this is without ec2 plugin and any gateway. now i want to configure
> > > > a s3 gateway on top of it. but if i follow the steps in the tutorial, it
> > > > ends up deleting my index! how do i get around this problem? is it even
> > > > possible?
> > > >
> > > > i took a copy of the data as a backup and hence i am able to retry
> > > > multiple times by restoring the data back..
> > > >
> > > > thanks
> > > >
> > > > ps - i followed the steps in the tutorial with a new cluster and it
> > > > works perfectly. also, for my experiments, i am renaming the cluster by
> > > > simply renaming the cluster name dir in data dir.
---
### Author:  [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)
#### Post date: [July 18, 2012, 9:00pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/6 "2012-07-18T21:00:01Z")
🙂
i know the approach you mentioned would work.. but i am exploring s3 option
to save on cost. my current traffic doesnt really warrant 2 nodes. so i
want to be on 1 node (to save money). and i am also not inclined on ebs
because it would cost more and moreover, if i have 1 node, then i have to
worry about taking periodic snapshots to keep a backup.
so to save all this trouble and cost, i am interested in s3.
thanks
On Wed, Jul 18, 2012 at 1:52 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> It is clear, but you can also simply store it on EBS, or have more nodes
> with more copies of your data.
>
> On Wed, Jul 18, 2012 at 1:47 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
>
> > just to clarify, local default gateway is what you specify in "path.data"
> > setting right?
> >
> > what i want to achieve from s3 is to handle a scenario where my node that
> > uses local ephemeral disk as local data store and then due to some outage
> > or crash, the local data is lost/corrupted. i should be easily able to
> > create another instance and point it to s3 and then it restores all the
> > data locally on that machine and is able to serve data and index data..
> >
> > is that clear?
> >
> > thanks
> >
> > On Wed, Jul 18, 2012 at 1:41 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> >
> > > Its recommended to use the default local gateway even on AWS. No need to
> > > move to s3 gateway really.
> > >
> > > On Wed, Jul 18, 2012 at 12:21 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> > >
> > > > hi,
> > > > i tried different routes and still not able to get it to work. any
> > > > ideas?
> > > > i even tried copying the whole data dir structure on s3 and using the
> > > > same name as cluster name and yet didnt work.. ES would empty out the
> > > > cluster name dir.
> > > >
> > > > thanks
> > > >
> > > > On Tue, Jul 17, 2012 at 4:10 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> > > >
> > > > > hi,
> > > > > i have a running ES cluster (of 1 node only) on ec2 with lots of data
> > > > > on it.. this is without ec2 plugin and any gateway. now i want to configure
> > > > > a s3 gateway on top of it. but if i follow the steps in the tutorial, it
> > > > > ends up deleting my index! how do i get around this problem? is it even
> > > > > possible?
> > > > >
> > > > > i took a copy of the data as a backup and hence i am able to retry
> > > > > multiple times by restoring the data back..
> > > > >
> > > > > thanks
> > > > >
> > > > > ps - i followed the steps in the tutorial with a new cluster and it
> > > > > works perfectly. also, for my experiments, i am renaming the cluster by
> > > > > simply renaming the cluster name dir in data dir.
---
### Author:  [@T\_Vinod\_Gupta](https://discuss.elastic.co/u/T_Vinod_Gupta)
#### Post date: [July 18, 2012, 9:58pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/7 "2012-07-18T21:58:00Z")
another reason for not using EBS is that it is slower than local disk.
thanks
On Wed, Jul 18, 2012 at 2:00 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> 🙂
> i know the approach you mentioned would work.. but i am exploring s3
> option to save on cost. my current traffic doesnt really warrant 2 nodes.
> so i want to be on 1 node (to save money). and i am also not inclined on
> ebs because it would cost more and moreover, if i have 1 node, then i have
> to worry about taking periodic snapshots to keep a backup.
> so to save all this trouble and cost, i am interested in s3.
>
> thanks
>
> On Wed, Jul 18, 2012 at 1:52 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
>
> > It is clear, but you can also simply store it on EBS, or have more nodes
> > with more copies of your data.
> >
> > On Wed, Jul 18, 2012 at 1:47 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> >
> > > just to clarify, local default gateway is what you specify in
> > > "path.data" setting right?
> > >
> > > what i want to achieve from s3 is to handle a scenario where my node
> > > that uses local ephemeral disk as local data store and then due to some
> > > outage or crash, the local data is lost/corrupted. i should be easily able
> > > to create another instance and point it to s3 and then it restores all the
> > > data locally on that machine and is able to serve data and index data..
> > >
> > > is that clear?
> > >
> > > thanks
> > >
> > > On Wed, Jul 18, 2012 at 1:41 PM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote:
> > >
> > > > Its recommended to use the default local gateway even on AWS. No need
> > > > to move to s3 gateway really.
> > > >
> > > > On Wed, Jul 18, 2012 at 12:21 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> > > >
> > > > > hi,
> > > > > i tried different routes and still not able to get it to work. any
> > > > > ideas?
> > > > > i even tried copying the whole data dir structure on s3 and using the
> > > > > same name as cluster name and yet didnt work.. ES would empty out the
> > > > > cluster name dir.
> > > > >
> > > > > thanks
> > > > >
> > > > > On Tue, Jul 17, 2012 at 4:10 PM, T Vinod Gupta [tvinod@readypulse.com](mailto:tvinod@readypulse.com)wrote:
> > > > >
> > > > > > hi,
> > > > > > i have a running ES cluster (of 1 node only) on ec2 with lots of data
> > > > > > on it.. this is without ec2 plugin and any gateway. now i want to configure
> > > > > > a s3 gateway on top of it. but if i follow the steps in the tutorial, it
> > > > > > ends up deleting my index! how do i get around this problem? is it even
> > > > > > possible?
> > > > > >
> > > > > > i took a copy of the data as a backup and hence i am able to retry
> > > > > > multiple times by restoring the data back..
> > > > > >
> > > > > > thanks
> > > > > >
> > > > > > ps - i followed the steps in the tutorial with a new cluster and it
> > > > > > works perfectly. also, for my experiments, i am renaming the cluster by
> > > > > > simply renaming the cluster name dir in data dir.
---
### Author:  [@drewr](https://discuss.elastic.co/u/drewr)
#### Post date: [July 18, 2012, 10:50pm UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/8 "2012-07-18T22:50:10Z")
T Vinod Gupta wrote:
> another reason for not using EBS is that it is slower than local disk.
This is not always true, especially if you stripe your EBS volumes
appropriately. We've also had private talks with AWS about their
next gen EBS offering which will even further improve IOPS.
-Drew
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 3:19am UTC](https://discuss.elastic.co/t/how-to-add-ec2-s3-or-other-gateway-after-index-is-created/8443/9 "2017-07-06T03:19:53Z")