# Question about s3 gateway vs EBS **URL:** \ **Category:** Elasticsearch\ **Created:** [January 7, 2012, 12:08am UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307 "2012-01-07T00:08:26Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Gautam\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gautam_2/32/3025_2.png) [@Gautam\_2](https://discuss.elastic.co/u/Gautam_2)\ **Post date:** [January 7, 2012, 12:08am UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/1 "2012-01-07T00:08:26Z")
I'm about to set up a new cluster on ec2 and the idea of using the s3 gateway looks very appealing - if I can avoid dealing with EBS variable latencies, striping etc. it would be great. However, I got the sense from reading posts here and elsewhere that s3 is not the preferable solution if I am to expect reasonably high load on my cluster - is that the general feeling? If so I'm comfortable enough with EBS that I'd go with it instead. ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [January 7, 2012, 1:55am UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/2 "2012-01-07T01:55:08Z")
S3 will not help you alleviate any latency issues since its performance is no better than EBS. On Fri, Jan 6, 2012 at 4:08 PM, Gautam [golwala@gmail.com](mailto:golwala@gmail.com) wrote: > I'm about to set up a new cluster on ec2 and the idea of using the s3 > gateway looks very appealing - if I can avoid dealing with EBS variable > latencies, striping etc. it would be great. However, I got the sense from > reading posts here and elsewhere that s3 is not the preferable solution if I > am to expect reasonably high load on my cluster - is that the general > feeling? If so I'm comfortable enough with EBS that I'd go with it instead. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 7, 2012, 7:37pm UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/3 "2012-01-07T19:37:22Z")
Yea, s3 basically just snapshots the current state of the local stored indexes, you still need to have the index be stored locally. It can be ephemeral drives or EBS. On Sat, Jan 7, 2012 at 3:55 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > S3 will not help you alleviate any latency issues since its > performance is no better than EBS. > > On Fri, Jan 6, 2012 at 4:08 PM, Gautam [golwala@gmail.com](mailto:golwala@gmail.com) wrote: > > > I'm about to set up a new cluster on ec2 and the idea of using the s3 > > gateway looks very appealing - if I can avoid dealing with EBS variable > > latencies, striping etc. it would be great. However, I got the sense from > > reading posts here and elsewhere that s3 is not the preferable solution > > if I > > am to expect reasonably high load on my cluster - is that the general > > feeling? If so I'm comfortable enough with EBS that I'd go with it > > instead. ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [January 8, 2012, 6:52am UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/4 "2012-01-08T06:52:35Z")
Shay, is there any "definitive guide" explaining the differences between S3 and EBS based persistence? The way I see it, it's about: a) S3 is easy to setup and use, it works "out of the box" b) recreating the state of a large cluster will be painful from S3, since the data must be physically copied to machines; with EBS, the data is "already there" Correct? Thanks!, Karel On Jan 7, 8:37 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > Yea, s3 basically just snapshots the current state of the local stored > indexes, you still need to have the index be stored locally. It can > be ephemeral drives or EBS. > > On Sat, Jan 7, 2012 at 3:55 AM, Ivan Brusic [i...@brusic.com](mailto:i...@brusic.com) wrote: > > > S3 will not help you alleviate any latency issues since its > > performance is no better than EBS. > > > On Fri, Jan 6, 2012 at 4:08 PM, Gautam [golw...@gmail.com](mailto:golw...@gmail.com) wrote: > > > > > I'm about to set up a new cluster on ec2 and the idea of using the s3 > > > gateway looks very appealing - if I can avoid dealing with EBS variable > > > latencies, striping etc. it would be great. However, I got the sense from > > > reading posts here and elsewhere that s3 is not the preferable solution > > > if I > > > am to expect reasonably high load on my cluster - is that the general > > > feeling? If so I'm comfortable enough with EBS that I'd go with it > > > instead. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 8, 2012, 1:51pm UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/5 "2012-01-08T13:51:37Z")
Yea, b) is the important part. Also, while the system is running, there is a need to keep updating s3 with the current state of the index (in a shared gateway mode). In the future, I hope to combine the two. Allow to use local gateway (EBS or not), and allow to snapshot (whenever you want) the state to s3, and recover from it if needed to. On Sun, Jan 8, 2012 at 8:52 AM, Karel Minařík [karel.minarik@gmail.com](mailto:karel.minarik@gmail.com)wrote: > Shay, is there any "definitive guide" explaining the differences > between S3 and EBS based persistence? > > The way I see it, it's about: > > a) S3 is easy to setup and use, it works "out of the box" > b) recreating the state of a large cluster will be painful from S3, > since the data must be physically copied to machines; with EBS, the > data is "already there" > > Correct? > > Thanks!, > > Karel > > On Jan 7, 8:37 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > Yea, s3 basically just snapshots the current state of the local stored > > indexes, you still need to have the index be stored locally. It can > > be ephemeral drives or EBS. > > > > On Sat, Jan 7, 2012 at 3:55 AM, Ivan Brusic [i...@brusic.com](mailto:i...@brusic.com) wrote: > > > > > S3 will not help you alleviate any latency issues since its > > > performance is no better than EBS. > > > > > On Fri, Jan 6, 2012 at 4:08 PM, Gautam [golw...@gmail.com](mailto:golw...@gmail.com) wrote: > > > > > > > I'm about to set up a new cluster on ec2 and the idea of using the s3 > > > > gateway looks very appealing - if I can avoid dealing with EBS > > > > variable > > > > latencies, striping etc. it would be great. However, I got the sense > > > > from > > > > reading posts here and elsewhere that s3 is not the preferable > > > > solution > > > > if I > > > > am to expect reasonably high load on my cluster - is that the general > > > > feeling? If so I'm comfortable enough with EBS that I'd go with it > > > > instead. ---
**Author:** ![James\_Cook\_3](https://avatars.discourse-cdn.com/v4/letter/j/d2c977/32.png) [@James\_Cook\_3](https://discuss.elastic.co/u/James_Cook_3)\ **Post date:** [January 9, 2012, 4:36am UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/6 "2012-01-09T04:36:12Z")
Or memory ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [January 9, 2012, 8:02pm UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/7 "2012-01-09T20:02:44Z")
You mean indices that are memory based? Yea, in this case, teh s3 gateway option is pretty cool since you can store the index completely in memory, but still be able to recover if nodes fail (even if all replicas are gone). On Mon, Jan 9, 2012 at 6:36 AM, James Cook [jcook@pykl.com](mailto:jcook@pykl.com) wrote: > Or memory ---
**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\ **Post date:** [January 10, 2012, 8:11am UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/8 "2012-01-10T08:11:24Z")
Yeah, the memory option would be a pretty powerful option for some well specified cases! At the moment, we're using S3 on AWS; yesterday I did a quick test and I was _very_ satisfied with the recovery speed. The setup: - One index, ~600,000 docs - 6.5GB The full recovery cycle -- launch two m1.large nodes (\<1min), bootstrap and provision them with Chef (\<3min) to "green" -- state took something like 10 minutes. Effectively ~1GB per minute. That's enough "realtime" in my book 🙂 Karel On Jan 9, 9:02 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > You mean indices that are memory based? Yea, in this case, teh s3 gateway > option is pretty cool since you can store the index completely in memory, > but still be able to recover if nodes fail (even if all replicas are gone). > > On Mon, Jan 9, 2012 at 6:36 AM, James Cook [jc...@pykl.com](mailto:jc...@pykl.com) wrote: > > > Or memory ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:43am UTC](https://discuss.elastic.co/t/question-about-s3-gateway-vs-ebs/6307/9 "2017-07-06T03:43:18Z")