# Struggling with EC2 Discovery

**URL:** <https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021>\
**Category:** Elasticsearch\
**Created:** [March 5, 2013, 5:54pm UTC](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021 "2013-03-05T17:54:35Z")\
**Posts on this page:** 6\
**Page:** 2

<div class="post-metadata">

**Author:** ![inZania](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inzania/32/2305_2.png) [@inZania](https://discuss.elastic.co/u/inZania)\
**Post date:** [March 6, 2013, 5:08am UTC](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021/21 "2013-03-06T05:08:06Z")

</div>

I still can't seem to get anywhere with this. Unicast works fine; no such luck with EC2. I'm so desperate I'm considering rolling my own script to detect the nodes and write them into the config file...

Note that there is a comment in the tutorial with the exact same problem from 5 months ago, but nobody mentions a solution.

---

<div class="post-metadata">

**Author:** ![inZania](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/inzania/32/2305_2.png) [@inZania](https://discuss.elastic.co/u/inZania)\
**Post date:** [March 6, 2013, 5:45am UTC](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021/22 "2013-03-06T05:45:34Z")

</div>

Ahhh, finally. Got it, thanks to this: [http://elasticsearch-users.115913.n3.nabble.com/EC2-discovery-issue-td3138803i20.html](http://elasticsearch-users.115913.n3.nabble.com/EC2-discovery-issue-td3138803i20.html)

For me, too, I had to set the cloud.aws.region: us-west-1

---

<div class="post-metadata">

**Author:** ![Craig\_Brown](https://avatars.discourse-cdn.com/v4/letter/c/ce7236/32.png) [@Craig\_Brown](https://discuss.elastic.co/u/Craig_Brown)\
**Post date:** [March 6, 2013, 7:40pm UTC](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021/23 "2013-03-06T19:40:04Z")

</div>

Great news! I've never had to use that setting for us-east. Once you get it  
to work, it really does well.

- Craig

On Tue, Mar 5, 2013 at 10:45 PM, inZania [zaneclaes@gmail.com](mailto:zaneclaes@gmail.com) wrote:

> Ahhh, finally. Got it, thanks to this:
> 
> [http://elasticsearch-users.115913.n3.nabble.com/EC2-discovery-issue-td3138803i20.html](http://elasticsearch-users.115913.n3.nabble.com/EC2-discovery-issue-td3138803i20.html)
> 
> For me, too, I had to set the cloud.aws.region: us-west-1
> 
> --  
> View this message in context:  
> [http://elasticsearch-users.115913.n3.nabble.com/Struggling-with-EC2-Discovery-tp4031058p4031106.html](http://elasticsearch-users.115913.n3.nabble.com/Struggling-with-EC2-Discovery-tp4031058p4031106.html)  
> Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
…  
CRAIG BROWN  
Search Architect  
youwho, Inc.

[www.youwho.com](http://www.youwho.com)

M: 801.680.7300

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![karmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/karmi/32/44951_2.png) [@karmi](https://discuss.elastic.co/u/karmi)\
**Post date:** [March 7, 2013, 7:25am UTC](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021/24 "2013-03-07T07:25:34Z")

</div>

> Ahhh, finally. (...)  
> For me, too, I had to set the cloud.aws.region: us-west-1

You have to set the region if you're not using the default `us-east-1`.  
Filtering the nodes by a specific security group as suggested by Paul above  
is correct and the easiest way how to detect Elasticsearch nodes in  
discovery. Note also, that currently, you don't have to put the AWS  
credentials into the elasticsearch.yml file, if you create the instance  
with a IAM role. See

> **[IAM FAQs](https://aws.amazon.com/iam/faqs/#How_do_i_get_started_with_IAM_roles_for_EC2_instances)**

> to answer the second part of your question, there is no data loss as  
> part of autoscaling. the shards will just be re-allocated. you can  
> autoscale to your heart's content.

This is a bit dangerous advice .) You _will_ lose data if you or your  
autoscaling mechanism adds nodes to the cluster and terminates them later,  
_unless_ you increase the `number_of_replicas` for your indices. There's a  
`index.auto_expand_replicas` which would make your index shrink/grow in  
sync with the number of nodes in your cluster.

Karel

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Paul\_Sanwald\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/paul_sanwald_2/32/881_2.png) [@Paul\_Sanwald\_2](https://discuss.elastic.co/u/Paul_Sanwald_2)\
**Post date:** [March 7, 2013, 11:40am UTC](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021/25 "2013-03-07T11:40:09Z")

</div>

Karel, you are right that I should have qualified my statement: you can  
autoscale as you like as long as your minimum number of nodes can support  
your index size.

My autoscaling is setup for load based scaling (a lot of queries at  
once) as opposed to handling increases in data, so I didn't think about the  
latter right away. I have the luxury of a relatively static index (for now).

On Thursday, March 7, 2013, Karel Minařík wrote:

> Ahhh, finally. (...)
> 
> > For me, too, I had to set the cloud.aws.region: us-west-1
> 
> You have to set the region if you're not using the default `us-east-1`.  
> Filtering the nodes by a specific security group as suggested by Paul above  
> is correct and the easiest way how to detect Elasticsearch nodes in  
> discovery. Note also, that currently, you don't have to put the AWS  
> credentials into the elasticsearch.yml file, if you create the instance  
> with a IAM role. See  
> [IAM FAQs](http://aws.amazon.com/iam/faqs/#How_do_i_get_started_with_IAM_roles_for_EC2_instances)
> 
> > to answer the second part of your question, there is no data loss as  
> > part of autoscaling. the shards will just be re-allocated. you can  
> > autoscale to your heart's content.
> 
> This is a bit dangerous advice .) You _will_ lose data if you or your  
> autoscaling mechanism adds nodes to the cluster and terminates them later,  
> _unless_ you increase the `number_of_replicas` for your indices. There's a  
> `index.auto_expand_replicas` which would make your index shrink/grow in  
> sync with the number of nodes in your cluster.
> 
> Karel
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/RG7yEev0Zc8/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/RG7yEev0Zc8/unsubscribe?hl=en-US)  
> .  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com) \<javascript:\_e({}, 'cvml',  
> 'elasticsearch%2Bunsubscribe@googlegroups.com');\>.  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:47am UTC](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021/26 "2017-07-06T02:47:47Z")

</div>

Closed on July 6, 2017, 2:47am UTC



[Previous page](https://discuss.elastic.co/t/struggling-with-ec2-discovery/11021.md?page=1)
