Problems connecting with other nodes

I'm stumped. I'm sure I've got something misconfigured, but I can't figure
it out.

I am trying to setup a new cluster of two nodes running 0.20.6 on EC2. The
EC2 Discovery phase works in that it finds the other potential nodes in the
cluster. BUT, the two nodes won't talk to each other. Basically, ES fails
to connect to the other node(s) on port 9300:

[2013-03-29 11:04:16,642][WARN ][discovery.zen.ping.unicast] [
es3.logstash.ec2.example.com] failed to send ping to
[[#cloud-i-12345678-0][inet[/a.b.c.d:9300]]]
org.elasticsearch.transport.ReceiveTimeoutTransportException:
[][inet[/a.b.c.d:9300]][discovery/zen/unicast] request_id [4] timed out
after [3750ms]

As far as I can tell, the problem is that the nodes are not binding to port
9300. Instead they're binding to 9302:

2013-03-29 11:04:09,250][INFO ][transport ] [
es3.logstash.ec2.example.com] bound_address {inet[/0.0.0.0:9302]},
publish_address {inet[/aa.bb.cc.dd:9302]}

If have verified that the security group allows traffic on the port range
9200-9400. If I try to telnet from one of the two nodes to the other on
port 9300, telnet hangs. If I try to telnet to port 9302, it works. I also
tried disabling EC2 discovery and explicitly listing the nodes using port
9302 (instead of the default 9300) and that worked great:

discovery.zen.ping.unicast.hosts: ["es3.logstash.ec2.example.com:9302","
es4.logstash.ec2.example.com:9302"]

If I try explicitly forcing the transport port to 9300 and the http port to
9200, ElasticSearch complains that the ports are already in use and shuts
back down.

If I use unicast, set the host names directly in the config and specify
port 9302, it works fine and the two nodes talk to each other.

My discovery configuration looks like the following:

discovery.type: ec2
discovery.zen.minimum_master_nodes: 1
discovery.zen.ping.multicast.enabled: false
cloud.node.auto_attributes: true
cloud.aws.region: us-east-1
discovery.ec2.groups: ElasticSearchLogstash

Any bright ideas? I'm pretty much dead in the water until I can get this
working...

-Sean

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Hey,

this looks like there is already another service running on port 9300 and
9200. If you are on linux you can verify this with 'lsof' or 'netstat
-tulpe' (as root). Maybe you started an elasticsearch instance already with
different configurations some time ago.

On Thu, Apr 4, 2013 at 2:49 AM, VegHead organicveggie@gmail.com wrote:

I'm stumped. I'm sure I've got something misconfigured, but I can't figure
it out.

I am trying to setup a new cluster of two nodes running 0.20.6 on EC2. The
EC2 Discovery phase works in that it finds the other potential nodes in the
cluster. BUT, the two nodes won't talk to each other. Basically, ES fails
to connect to the other node(s) on port 9300:

[2013-03-29 11:04:16,642][WARN ][discovery.zen.ping.unicast] [
es3.logstash.ec2.example.com] failed to send ping to
[[#cloud-i-12345678-0][inet[/**a.b.c.d:9300]]]
org.elasticsearch.transport.ReceiveTimeoutTransportException:
[][inet[/a.b.c.d:9300]][**discovery/zen/unicast] request_id [4] timed out
after [3750ms]

As far as I can tell, the problem is that the nodes are not binding to
port 9300. Instead they're binding to 9302:

2013-03-29 11:04:09,250][INFO ][transport ] [
es3.logstash.ec2.example.com] bound_address {inet[/0.0.0.0:9302]},
publish_address {inet[/aa.bb.cc.dd:9302]}

If have verified that the security group allows traffic on the port range
9200-9400. If I try to telnet from one of the two nodes to the other on
port 9300, telnet hangs. If I try to telnet to port 9302, it works. I also
tried disabling EC2 discovery and explicitly listing the nodes using port
9302 (instead of the default 9300) and that worked great:

discovery.zen.ping.unicast.**hosts: ["es3.logstash.ec2.example.**com:9302http://es3.logstash.ec2.example.com:9302/
","es4.logstash.ec2.**example.com:9302http://es4.logstash.ec2.example.com:9302/
"]

If I try explicitly forcing the transport port to 9300 and the http port
to 9200, ElasticSearch complains that the ports are already in use and
shuts back down.

If I use unicast, set the host names directly in the config and specify
port 9302, it works fine and the two nodes talk to each other.

My discovery configuration looks like the following:

discovery.type: ec2
discovery.zen.minimum_master_**nodes: 1
discovery.zen.ping.multicast.**enabled: false
cloud.node.auto_attributes: true
cloud.aws.region: us-east-1
discovery.ec2.groups: ElasticSearchLogstash

Any bright ideas? I'm pretty much dead in the water until I can get this
working...

-Sean

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.