# Running on EC2 S3 vs EBS **URL:** \ **Category:** Elasticsearch\ **Created:** [October 13, 2014, 9:17pm UTC](https://discuss.elastic.co/t/running-on-ec2-s3-vs-ebs/20227 "2014-10-13T21:17:32Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Matthias\_Johnson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/matthias_johnson/32/1196_2.png) [@Matthias\_Johnson](https://discuss.elastic.co/u/Matthias_Johnson)\ **Post date:** [October 13, 2014, 9:17pm UTC](https://discuss.elastic.co/t/running-on-ec2-s3-vs-ebs/20227/1 "2014-10-13T21:17:32Z")
We've begun deploying to AWS EC2. I've seen refrences in the group about the S3 gateway and it being deprecated. That seems to be confirmed by looking at the docs, which don't seem to list the S3 Gateway specifically after 0.90.x. We are also using the elasticsearch-cloud-aws plugins [https://github.com/elasticsearch/elasticsearch-cloud-aws](https://github.com/elasticsearch/elasticsearch-cloud-aws), which does a nice job at helping the auto discovery. It also shows settings for using S3. After some reading my understanding is that the plugin is basically just snapshots that are stored in S3. Is that understanding correct? Is this much different from the original gateway? That suggests that unless we take frequent snapshots we would run a risk of data loss if the entire cluster wen't down (right now we are using instance storage). Is that right? Switching to EBS would give us better protection against data loss, since the data is stored on a more permanent basis as well as improved recovery after an entire cluster going down? Are there any good guides on configuring this sort of setup with cloudformation and templates and/or tying EBS volumes for ES use to machines when a cluster is resurrected? @matthias -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\ **Post date:** [October 13, 2014, 9:36pm UTC](https://discuss.elastic.co/t/running-on-ec2-s3-vs-ebs/20227/2 "2014-10-13T21:36:47Z")
Yes, you don't want to use anything other than local storage for Elasticsearch. Not EBS and definitely not S3. You can use the snapshot/restore API to continously backup to S3 and get all the data protection you need. -- Itamar Syn-Hershko [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) Freelance Developer & Consultant Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) On Tue, Oct 14, 2014 at 12:17 AM, Matthias Johnson [opennomad@gmail.com](mailto:opennomad@gmail.com) wrote: > We've begun deploying to AWS EC2. I've seen refrences in the group about > the S3 gateway and it being deprecated. That seems to be confirmed by > looking at the docs, which don't seem to list the S3 Gateway specifically > after 0.90.x. > > We are also using the elasticsearch-cloud-aws plugins > [https://github.com/elasticsearch/elasticsearch-cloud-aws](https://github.com/elasticsearch/elasticsearch-cloud-aws), which does a > nice job at helping the auto discovery. It also shows settings for using S3. > > After some reading my understanding is that the plugin is basically just > snapshots that are stored in S3. Is that understanding correct? Is this > much different from the original gateway? > > That suggests that unless we take frequent snapshots we would run a risk > of data loss if the entire cluster wen't down (right now we are using > instance storage). Is that right? > > Switching to EBS would give us better protection against data loss, since > the data is stored on a more permanent basis as well as improved recovery > after an entire cluster going down? > > Are there any good guides on configuring this sort of setup with > cloudformation and templates and/or tying EBS volumes for ES use to > machines when a cluster is resurrected? > > @matthias > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtPqGN7PDTQkoYcsYwfM\_3bmVrEECwZcCiNqDsLsa9gqQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtPqGN7PDTQkoYcsYwfM_3bmVrEECwZcCiNqDsLsa9gqQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Norberto\_Meijome](https://avatars.discourse-cdn.com/v4/letter/n/ed655f/32.png) [@Norberto\_Meijome](https://discuss.elastic.co/u/Norberto_Meijome)\ **Post date:** [October 14, 2014, 2:21am UTC](https://discuss.elastic.co/t/running-on-ec2-s3-vs-ebs/20227/3 "2014-10-14T02:21:51Z")
Or, if your use case allows for it, have a very well oiled rebuild process (data included). On 14/10/2014 8:36 am, "Itamar Syn-Hershko" [itamar@code972.com](mailto:itamar@code972.com) wrote: > Yes, you don't want to use anything other than local storage for > Elasticsearch. Not EBS and definitely not S3. You can use the > snapshot/restore API to continously backup to S3 and get all the data > protection you need. > > -- > > Itamar Syn-Hershko > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko) > Freelance Developer & Consultant > Author of RavenDB in Action [http://manning.com/synhershko/](http://manning.com/synhershko/) > > On Tue, Oct 14, 2014 at 12:17 AM, Matthias Johnson [opennomad@gmail.com](mailto:opennomad@gmail.com) > wrote: > > > We've begun deploying to AWS EC2. I've seen refrences in the group about > > the S3 gateway and it being deprecated. That seems to be confirmed by > > looking at the docs, which don't seem to list the S3 Gateway specifically > > after 0.90.x. > > > > We are also using the elasticsearch-cloud-aws plugins > > [https://github.com/elasticsearch/elasticsearch-cloud-aws](https://github.com/elasticsearch/elasticsearch-cloud-aws), which does a > > nice job at helping the auto discovery. It also shows settings for using S3. > > > > After some reading my understanding is that the plugin is basically just > > snapshots that are stored in S3. Is that understanding correct? Is this > > much different from the original gateway? > > > > That suggests that unless we take frequent snapshots we would run a risk > > of data loss if the entire cluster wen't down (right now we are using > > instance storage). Is that right? > > > > Switching to EBS would give us better protection against data loss, since > > the data is stored on a more permanent basis as well as improved recovery > > after an entire cluster going down? > > > > Are there any good guides on configuring this sort of setup with > > cloudformation and templates and/or tying EBS volumes for ES use to > > machines when a cluster is resurrected? > > > > @matthias > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/415a7669-4c2a-4d3e-a960-67390c1197cf%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtPqGN7PDTQkoYcsYwfM\_3bmVrEECwZcCiNqDsLsa9gqQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtPqGN7PDTQkoYcsYwfM_3bmVrEECwZcCiNqDsLsa9gqQ%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtPqGN7PDTQkoYcsYwfM\_3bmVrEECwZcCiNqDsLsa9gqQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZtPqGN7PDTQkoYcsYwfM_3bmVrEECwZcCiNqDsLsa9gqQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CACj2-4%2BQqbg9\_PF7i-timWB14\_nrZ5asFj3%3DXxk5Wd8qabZvcw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CACj2-4%2BQqbg9_PF7i-timWB14_nrZ5asFj3%3DXxk5Wd8qabZvcw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:56am UTC](https://discuss.elastic.co/t/running-on-ec2-s3-vs-ebs/20227/4 "2017-07-06T00:56:28Z")