Having a problem getting ec2_discovery to work in us_west

I have been able to get everything up and running perfectly in the
Amazon us-east region. The config I'm using is leveraging the groups
for the two nodes two find each other.

I have what I think is an identical setup in the us-west region and
the two nodes are not seeing each other - they both startup as
master. I've tried a number of other things like setting the
availability zone (to us-west-1a), changing the host type, etc., but I
can't seem to get it to work.

Is there something I should be looking for and/or configuring
differently for the two regions? Should this work in us-west? Is
there any other information I can post to help you guys in helping me
troubleshoot the issue?

Can you set the logging for discovery.ec2 to TRACE? Lets see when if those
nodes gets discovered. Do you see anything in the logs?

On Wed, Dec 22, 2010 at 11:47 PM, Gary B. garybenitt@gmail.com wrote:

I have been able to get everything up and running perfectly in the
Amazon us-east region. The config I'm using is leveraging the groups
for the two nodes two find each other.

I have what I think is an identical setup in the us-west region and
the two nodes are not seeing each other - they both startup as
master. I've tried a number of other things like setting the
availability zone (to us-west-1a), changing the host type, etc., but I
can't seem to get it to work.

Is there something I should be looking for and/or configuring
differently for the two regions? Should this work in us-west? Is
there any other information I can post to help you guys in helping me
troubleshoot the issue?

Yeah, it's pretty much what I thought. It ends up pulling up my groups from
us-east instead of us-west - so it's not surprise that there's no match...

I'm guessing that this request isn't specifying the region: Sending Request:
POST http://ec2.amazonaws.com ? I would think that you would probably need
to know that I'm trying to access us-west with another param and then use
the appropriate endpoint for that region (ec2.us-west-1.amazonaws.com)?

On Wed, Dec 22, 2010 at 8:38 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Can you set the logging for discovery.ec2 to TRACE? Lets see when if those
nodes gets discovered. Do you see anything in the logs?

On Wed, Dec 22, 2010 at 11:47 PM, Gary B. garybenitt@gmail.com wrote:

I have been able to get everything up and running perfectly in the
Amazon us-east region. The config I'm using is leveraging the groups
for the two nodes two find each other.

I have what I think is an identical setup in the us-west region and
the two nodes are not seeing each other - they both startup as
master. I've tried a number of other things like setting the
availability zone (to us-west-1a), changing the host type, etc., but I
can't seem to get it to work.

Is there something I should be looking for and/or configuring
differently for the two regions? Should this work in us-west? Is
there any other information I can post to help you guys in helping me
troubleshoot the issue?

Heya,

You can set the endpoint using the cloud.aws.endpoint setting. But, after
looking at this a bit more closely, this setting is also used for s3 gateway
(not sure if you use it), so they clash. In 0.14, this setting will change
to cloud.aws.ec2.endpoint (for the discovery part), but, there will also be
a simplified setting called cloud.aws.region that will automatically set
things correctly for s3 and ec2, as well as s3 region setting when creating
a bucket. Here is the issue:
Cloud AWS: Change endpoint parameters to distinguish between ec2 and s3 · Issue #574 · elastic/elasticsearch · GitHub.

-shay.banon

On Thu, Dec 23, 2010 at 7:44 AM, Gary Benitt garybenitt@gmail.com wrote:

Yeah, it's pretty much what I thought. It ends up pulling up my groups
from us-east instead of us-west - so it's not surprise that there's no
match...

I'm guessing that this request isn't specifying the region: Sending
Request: POST http://ec2.amazonaws.com ? I would think that you would
probably need to know that I'm trying to access us-west with another param
and then use the appropriate endpoint for that region (
ec2.us-west-1.amazonaws.com)?

On Wed, Dec 22, 2010 at 8:38 PM, Shay Banon shay.banon@elasticsearch.comwrote:

Can you set the logging for discovery.ec2 to TRACE? Lets see when if those
nodes gets discovered. Do you see anything in the logs?

On Wed, Dec 22, 2010 at 11:47 PM, Gary B. garybenitt@gmail.com wrote:

I have been able to get everything up and running perfectly in the
Amazon us-east region. The config I'm using is leveraging the groups
for the two nodes two find each other.

I have what I think is an identical setup in the us-west region and
the two nodes are not seeing each other - they both startup as
master. I've tried a number of other things like setting the
availability zone (to us-west-1a), changing the host type, etc., but I
can't seem to get it to work.

Is there something I should be looking for and/or configuring
differently for the two regions? Should this work in us-west? Is
there any other information I can post to help you guys in helping me
troubleshoot the issue?

I'm not currently using the s3 gateway functionality so this solution works
just fine for me... thanks for pointing it out...

And as you say, the simplified setting for region along with the specific
endpoints makes a lot of sense...

Thanks again for the help...

On Wed, Dec 22, 2010 at 11:16 PM, Shay Banon
shay.banon@elasticsearch.comwrote:

Heya,

You can set the endpoint using the cloud.aws.endpoint setting. But, after
looking at this a bit more closely, this setting is also used for s3 gateway
(not sure if you use it), so they clash. In 0.14, this setting will change
to cloud.aws.ec2.endpoint (for the discovery part), but, there will also be
a simplified setting called cloud.aws.region that will automatically set
things correctly for s3 and ec2, as well as s3 region setting when creating
a bucket. Here is the issue:
Cloud AWS: Change endpoint parameters to distinguish between ec2 and s3 · Issue #574 · elastic/elasticsearch · GitHub.

-shay.banon

On Thu, Dec 23, 2010 at 7:44 AM, Gary Benitt garybenitt@gmail.com wrote:

Yeah, it's pretty much what I thought. It ends up pulling up my groups
from us-east instead of us-west - so it's not surprise that there's no
match...

I'm guessing that this request isn't specifying the region: Sending
Request: POST http://ec2.amazonaws.com ? I would think that you would
probably need to know that I'm trying to access us-west with another param
and then use the appropriate endpoint for that region (
ec2.us-west-1.amazonaws.com)?

On Wed, Dec 22, 2010 at 8:38 PM, Shay Banon <shay.banon@elasticsearch.com

wrote:

Can you set the logging for discovery.ec2 to TRACE? Lets see when if
those nodes gets discovered. Do you see anything in the logs?

On Wed, Dec 22, 2010 at 11:47 PM, Gary B. garybenitt@gmail.com wrote:

I have been able to get everything up and running perfectly in the
Amazon us-east region. The config I'm using is leveraging the groups
for the two nodes two find each other.

I have what I think is an identical setup in the us-west region and
the two nodes are not seeing each other - they both startup as
master. I've tried a number of other things like setting the
availability zone (to us-west-1a), changing the host type, etc., but I
can't seem to get it to work.

Is there something I should be looking for and/or configuring
differently for the two regions? Should this work in us-west? Is
there any other information I can post to help you guys in helping me
troubleshoot the issue?