# Running Elasticsearch on AWS EC2

**URL:** https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763
**Category:** Elasticsearch
**Created:** [November 15, 2014, 8:01am UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763 "2014-11-15T08:01:20Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Tomer\_Levy](https://avatars.discourse-cdn.com/v4/letter/t/85e7bf/32.png) [@Tomer\_Levy](https://discuss.elastic.co/u/Tomer_Levy)
#### Post date: [November 15, 2014, 8:01am UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763/1 "2014-11-15T08:01:20Z")

</div>

Hello,

I'm trying to run Elasticsearch on EC2. After watching the webinar on  
running Elasticsearch on EC2, It was mentioned that the only working  
solution is using ephemeral storage since EBS is not fast enough.  
However, the current EC2 instance M3.large or even m3.xlarge come with very  
small disks that won't be enough (40GB). I want to ship 25GB a day and  
retain data for 30 days.

How do you recommend that I go about setting it up? Has anyone successfully  
used EBS?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [November 15, 2014, 8:57am UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763/2 "2014-11-15T08:57:04Z")

</div>

If you use EBS, then ask for provisioned IO.

HTH

David

> Le 15 nov. 2014 à 09:01, Tomer Levy [tomer.levy9@gmail.com](mailto:tomer.levy9@gmail.com) a écrit :
> 
> Hello,
> 
> I'm trying to run Elasticsearch on EC2. After watching the webinar on running Elasticsearch on EC2, It was mentioned that the only working solution is using ephemeral storage since EBS is not fast enough.  
> However, the current EC2 instance M3.large or even m3.xlarge come with very small disks that won't be enough (40GB). I want to ship 25GB a day and retain data for 30 days.
> 
> How do you recommend that I go about setting it up? Has anyone successfully used EBS?
> 
> --  
> You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com).  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6770DA70-03DE-4AB8-B69A-3A80456AE0D9%40pilato.fr](https://groups.google.com/d/msgid/elasticsearch/6770DA70-03DE-4AB8-B69A-3A80456AE0D9%40pilato.fr).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![davidski](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidski/32/44884_2.png) [@davidski](https://discuss.elastic.co/u/davidski)
#### Post date: [November 15, 2014, 2:30pm UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763/3 "2014-11-15T14:30:20Z")

</div>

With the relatively recent transition of default EBS from magnetic to  
SSD-backed storage with burstable IO, I suggest you test out your workloads  
and see what meets your required performance levels. I've successfully run  
ES on slow magentic-EBS when I couldn't justify the cost of PIOPS+EBS  
optimized instances but your needs may be (read: probably are) different  
than mine.

David

On Saturday, November 15, 2014 12:57:18 AM UTC-8, David Pilato wrote:

> If you use EBS, then ask for provisioned IO.
> 
> HTH
> 
> David
> 
> Le 15 nov. 2014 à 09:01, Tomer Levy \<[tomer...@gmail.com](mailto:tomer...@gmail.com) \<javascript:\>\> a  
> écrit :
> 
> Hello,
> 
> I'm trying to run Elasticsearch on EC2. After watching the webinar on  
> running Elasticsearch on EC2, It was mentioned that the only working  
> solution is using ephemeral storage since EBS is not fast enough.  
> However, the current EC2 instance M3.large or even m3.xlarge come with  
> very small disks that won't be enough (40GB). I want to ship 25GB a day and  
> retain data for 30 days.
> 
> How do you recommend that I go about setting it up? Has anyone  
> successfully used EBS?
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b73a9c3e-2acd-461d-8d45-32fa4c900d59%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b73a9c3e-2acd-461d-8d45-32fa4c900d59%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![Tomer\_Levy](https://avatars.discourse-cdn.com/v4/letter/t/85e7bf/32.png) [@Tomer\_Levy](https://discuss.elastic.co/u/Tomer_Levy)
#### Post date: [November 15, 2014, 5:40pm UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763/4 "2014-11-15T17:40:09Z")

</div>

Thanks David. PIOPS will make this solution extremely expensive (in the  
thousands of $) so I'll have to skip it.

I'll try to test EBS SSD on my own. Wondered if someone has already done  
that and can share insights from their experience.

Thanks.

On Saturday, November 15, 2014 4:30:20 PM UTC+2, David Severski wrote:

> With the relatively recent transition of default EBS from magnetic to  
> SSD-backed storage with burstable IO, I suggest you test out your workloads  
> and see what meets your required performance levels. I've successfully run  
> ES on slow magentic-EBS when I couldn't justify the cost of PIOPS+EBS  
> optimized instances but your needs may be (read: probably are) different  
> than mine.
> 
> David
> 
> On Saturday, November 15, 2014 12:57:18 AM UTC-8, David Pilato wrote:
> 
> > If you use EBS, then ask for provisioned IO.
> > 
> > HTH
> > 
> > David
> > 
> > Le 15 nov. 2014 à 09:01, Tomer Levy [tomer...@gmail.com](mailto:tomer...@gmail.com) a écrit :
> > 
> > Hello,
> > 
> > I'm trying to run Elasticsearch on EC2. After watching the webinar on  
> > running Elasticsearch on EC2, It was mentioned that the only working  
> > solution is using ephemeral storage since EBS is not fast enough.  
> > However, the current EC2 instance M3.large or even m3.xlarge come with  
> > very small disks that won't be enough (40GB). I want to ship 25GB a day and  
> > retain data for 30 days.
> > 
> > How do you recommend that I go about setting it up? Has anyone  
> > successfully used EBS?
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/dd0eb2fe-8113-424d-8634-6cb1c033dc0b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/dd0eb2fe-8113-424d-8634-6cb1c033dc0b%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![simpsora](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simpsora/32/647_2.png) [@simpsora](https://discuss.elastic.co/u/simpsora)
#### Post date: [November 16, 2014, 4:30am UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763/5 "2014-11-16T04:30:40Z")

</div>

Do you have a particular response time requirement for your ES cluster?  
I'd suggest starting with the cheapest option available (magnetic EBS),  
then moving up the ladder to the more expensive options until you're happy  
with the speed/cost tradeoff. I've run ES clusters on magnetic EBS volumes  
with no problem (e.g. for low-to-moderate volume logstash/kibana/marvel  
clusters).

EBS is definitely slower than ephemeral (since it's just NAS), but I think  
the webinar was maybe a bit quick to discount it. EBS didn't work for them  
at their scale, but that doesn't mean it won't work for you!

Ross

On Sunday, 16 November 2014 04:40:09 UTC+11, Tomer Levy wrote:

> Thanks David. PIOPS will make this solution extremely expensive (in the  
> thousands of $) so I'll have to skip it.
> 
> I'll try to test EBS SSD on my own. Wondered if someone has already done  
> that and can share insights from their experience.
> 
> Thanks.
> 
> On Saturday, November 15, 2014 4:30:20 PM UTC+2, David Severski wrote:
> 
> > With the relatively recent transition of default EBS from magnetic to  
> > SSD-backed storage with burstable IO, I suggest you test out your workloads  
> > and see what meets your required performance levels. I've successfully run  
> > ES on slow magentic-EBS when I couldn't justify the cost of PIOPS+EBS  
> > optimized instances but your needs may be (read: probably are) different  
> > than mine.
> > 
> > David
> > 
> > On Saturday, November 15, 2014 12:57:18 AM UTC-8, David Pilato wrote:
> > 
> > > If you use EBS, then ask for provisioned IO.
> > > 
> > > HTH
> > > 
> > > David
> > > 
> > > Le 15 nov. 2014 à 09:01, Tomer Levy [tomer...@gmail.com](mailto:tomer...@gmail.com) a écrit :
> > > 
> > > Hello,
> > > 
> > > I'm trying to run Elasticsearch on EC2. After watching the webinar on  
> > > running Elasticsearch on EC2, It was mentioned that the only working  
> > > solution is using ephemeral storage since EBS is not fast enough.  
> > > However, the current EC2 instance M3.large or even m3.xlarge come with  
> > > very small disks that won't be enough (40GB). I want to ship 25GB a day and  
> > > retain data for 30 days.
> > > 
> > > How do you recommend that I go about setting it up? Has anyone  
> > > successfully used EBS?
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0d460d38-78a3-444b-9dd2-542f16015393%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .  
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/7c367ba8-bcc1-4e8a-af4f-b39e8dad493c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/7c367ba8-bcc1-4e8a-af4f-b39e8dad493c%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![moshe\_zada](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/moshe_zada/32/53780_2.png) [@moshe\_zada](https://discuss.elastic.co/u/moshe_zada)
#### Post date: [November 16, 2014, 9:03pm UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763/6 "2014-11-16T21:03:48Z")

</div>

You can try using [GitHub - stec-inc/EnhanceIO: EnhanceIO Open Source for Linux](https://github.com/stec-inc/EnhanceIO) which create block  
device with magnetic disks and SSDs as cache.

On Saturday, November 15, 2014 10:01:20 AM UTC+2, Tomer Levy wrote:

> Hello,
> 
> I'm trying to run Elasticsearch on EC2. After watching the webinar on  
> running Elasticsearch on EC2, It was mentioned that the only working  
> solution is using ephemeral storage since EBS is not fast enough.  
> However, the current EC2 instance M3.large or even m3.xlarge come with  
> very small disks that won't be enough (40GB). I want to ship 25GB a day and  
> retain data for 30 days.
> 
> How do you recommend that I go about setting it up? Has anyone  
> successfully used EBS?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/66202533-8419-470c-a560-b90379da636e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/66202533-8419-470c-a560-b90379da636e%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 12:49am UTC](https://discuss.elastic.co/t/running-elasticsearch-on-aws-ec2/20763/7 "2017-07-06T00:49:46Z")

</div>


