Instances wont see each other on AWS

Elastic Stack Elasticsearch
1

Hi using ES 1.6.0 and the latest AWS dicovery plugin 2.6.0

There is no errors reported in logs, but the nodes don't see each other.

I setup up 2 master nodes as follows...

bootstrap.mlockall: true
cluster.name: my-cluster
# And 02 for the second master
node.name: "ES MASTER 01"
node.data: false
node.master: true

plugin.mandatory: "cloud-aws"

cloud.aws.access_key: "AKI...Z7Q"
cloud.aws.secret_key: "gKW...nAO"
cloud.aws.region: "us-east"

discovery.zen.minimum_master_nodes: 1
discovery.type: "ec2"
discovery.ec2.groups: "Elastic Search"
discovery.ec2.ping_timeout: "30s"
discovery.ec2.availability_zones: "us-east-1a"

discovery.zen.ping.multicast.enabled: false

Here are the logs...

[2015-07-13 15:02:19,346][INFO ][node                     ] [ES MASTER 01] version[1.6.0], pid[2532], build[cdd3ac4/2015-06-09T13:36:34Z]
[2015-07-13 15:02:19,346][INFO ][node                     ] [ES MASTER 01] initializing ...
[2015-07-13 15:02:19,378][INFO ][plugins                  ] [ES MASTER 01] loaded [cloud-aws], sites []
[2015-07-13 15:02:19,440][INFO ][env                      ] [ES MASTER 01] using [1] data paths, mounts [[(C:)]], net usable_space [6.8gb], net total_space [29.9gb], types [NTFS]
[2015-07-13 15:02:26,461][INFO ][node                     ] [ES MASTER 01] initialized
[2015-07-13 15:02:26,461][INFO ][node                     ] [ES MASTER 01] starting ...
[2015-07-13 15:02:26,851][INFO ][transport                ] [ES MASTER 01] bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/172.30.0.123:9300]}
[2015-07-13 15:02:26,866][INFO ][discovery                ] [ES MASTER 01] my-cluster/SwhSDhiDQzq4pM8jkhIuzw
[2015-07-13 15:02:56,884][WARN ][discovery                ] [ES MASTER 01] waited for 30s and no initial state was set by the discovery
[2015-07-13 15:02:56,962][INFO ][http                     ] [ES MASTER 01] bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address {inet[/172.30.0.123:9200]}
[2015-07-13 15:02:56,962][INFO ][node                     ] [ES MASTER 01] started
[2015-07-13 15:03:13,455][INFO ][cluster.service          ] [ES MASTER 01] new_master [ES MASTER 01][SwhSDhiDQzq4pM8jkhIuzw][WIN-3Q4EH3B8H1O][inet[/172.30.0.123:9300]]{data=false, master=true}, reason: zen-disco-join (elected_as_master)
[2015-07-13 15:03:13,517][INFO ][gateway                  ] [ES MASTER 01] recovered [0] indices into cluster_state
2

Btw it's on Windows 2008 R2 in Amazon.

Does the security group have to have 9200 and 9300 open also?
Same for Windows firewall?

3

Added 9300-9400 in the security group.Seems to work...

5

Ok I have 1 node that canot see the rest... Is it because it doesn't have public IP?

6

I added public IP to the instance and it worked. Do we need to have public IPs? The instance is running in VPC with 172 address...

7

They shouldn't need public IPs, you just need to make sure 9200-9299 and 9300-9399 are allowed in your security groups.

8

Yeah it only seems the nodes can see each other if I put a public IP on the instance from the VPC console.

No clue why...

9

Can you telnet between them on 9200 or 9300 with ES running?

Generally the issue comes down to security group problems.

10

Ok I'll...

11

So yeah I can telnet between the boxes... But still no go...