Struggling with EC2 Discovery

Elastic Stack Elasticsearch
21

I still can't seem to get anywhere with this. Unicast works fine; no such luck with EC2. I'm so desperate I'm considering rolling my own script to detect the nodes and write them into the config file...

Note that there is a comment in the tutorial with the exact same problem from 5 months ago, but nobody mentions a solution.

22

Ahhh, finally. Got it, thanks to this: http://elasticsearch-users.115913.n3.nabble.com/EC2-discovery-issue-td3138803i20.html

For me, too, I had to set the cloud.aws.region: us-west-1

23

Great news! I've never had to use that setting for us-east. Once you get it
to work, it really does well.

  • Craig

On Tue, Mar 5, 2013 at 10:45 PM, inZania zaneclaes@gmail.com wrote:

Ahhh, finally. Got it, thanks to this:

http://elasticsearch-users.115913.n3.nabble.com/EC2-discovery-issue-td3138803i20.html

For me, too, I had to set the cloud.aws.region: us-west-1

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Struggling-with-EC2-Discovery-tp4031058p4031106.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
…
CRAIG BROWN
Search Architect
youwho, Inc.

www.youwho.com

M: 801.680.7300

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

24

Ahhh, finally. (...)
For me, too, I had to set the cloud.aws.region: us-west-1

You have to set the region if you're not using the default us-east-1.
Filtering the nodes by a specific security group as suggested by Paul above
is correct and the easiest way how to detect Elasticsearch nodes in
discovery. Note also, that currently, you don't have to put the AWS
credentials into the elasticsearch.yml file, if you create the instance
with a IAM role. See

to answer the second part of your question, there is no data loss as
part of autoscaling. the shards will just be re-allocated. you can
autoscale to your heart's content.

This is a bit dangerous advice .) You will lose data if you or your
autoscaling mechanism adds nodes to the cluster and terminates them later,
unless you increase the number_of_replicas for your indices. There's a
index.auto_expand_replicas which would make your index shrink/grow in
sync with the number of nodes in your cluster.

Karel

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

25

Karel, you are right that I should have qualified my statement: you can
autoscale as you like as long as your minimum number of nodes can support
your index size.

My autoscaling is setup for load based scaling (a lot of queries at
once) as opposed to handling increases in data, so I didn't think about the
latter right away. I have the luxury of a relatively static index (for now).

On Thursday, March 7, 2013, Karel Minařík wrote:

Ahhh, finally. (...)

For me, too, I had to set the cloud.aws.region: us-west-1

You have to set the region if you're not using the default us-east-1.
Filtering the nodes by a specific security group as suggested by Paul above
is correct and the easiest way how to detect Elasticsearch nodes in
discovery. Note also, that currently, you don't have to put the AWS
credentials into the elasticsearch.yml file, if you create the instance
with a IAM role. See
IAM FAQs

to answer the second part of your question, there is no data loss as
part of autoscaling. the shards will just be re-allocated. you can
autoscale to your heart's content.

This is a bit dangerous advice .) You will lose data if you or your
autoscaling mechanism adds nodes to the cluster and terminates them later,
unless you increase the number_of_replicas for your indices. There's a
index.auto_expand_replicas which would make your index shrink/grow in
sync with the number of nodes in your cluster.

Karel

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/RG7yEev0Zc8/unsubscribe?hl=en-US
.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com <javascript:_e({}, 'cvml',
'elasticsearch%2Bunsubscribe@googlegroups.com');>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.