[AWS, VPC] ElasticSearch 0.20.5 EC2 discovery doesn't work on EC2 VPC instance without Elastic IP

Hi there,

We are trying to setup ElasticSearch cluster on AWS, but unfortunately
private instances inside VPC (without Elastic IPs) are not able to start
correctly.
Here is our *elasticsearch.yml:

**
discovery.type: ec2
discovery.ec2.ping_timeout: 10s
discovery.zen.ping.multicast.enabled: false

Start log:
*
[2013-03-22 07:44:36,833][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: initializing ...
[2013-03-22 07:44:36,874][INFO ][plugins ] [Rom the
Spaceknight] loaded [cloud-aws], sites []
[2013-03-22
07:44:40,756][DEBUG][discovery.zen.ping.unicast] [Rom the Spaceknight]
using initial hosts [], with concurrent_connects [10]
[2013-03-22
07:44:40,758][DEBUG][discovery.ec2 ] [Rom the Spaceknight]
using ping.timeout [10s], master_election.filter_client [true],
master_election.filter_data [false]
[2013-03-22 07:44:40,762][DEBUG][discovery.zen.elect ] [Rom the
Spaceknight] using minimum_master_nodes [-1]
[2013-03-22
07:44:40,763][DEBUG][discovery.zen.fd ] [Rom the Spaceknight]
[master] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2013-03-22
07:44:40,768][DEBUG][discovery.zen.fd ] [Rom the Spaceknight]
[node ] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2013-03-22
07:44:41,436][DEBUG][discovery.ec2 ] [Rom the Spaceknight]
using host_type [PRIVATE_IP], tags [{}], groups [[]] with any_group
[true], availability_zones [[]]
[2013-03-22 07:44:44,107][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: initialized
[2013-03-22 07:44:44,107][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: starting ...
[2013-03-22
07:44:44,557][INFO ][transport ] [Rom the Spaceknight]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.0.0.108:9300]}
[2013-03-22
07:44:44,639][TRACE][discovery ] [Rom the Spaceknight]
waiting for 30s for the initial state to be set by the discovery
[2013-03-22
07:45:14,639][WARN ][discovery ] [Rom the Spaceknight]
waited for 30s and no initial state was set by the discovery
[2013-03-22 07:45:14,640][INFO ][discovery ] [Rom the
Spaceknight] ec2_vpc_private/JIIcSwi2SoKobfzXTeJCjg
[2013-03-22
07:45:14,647][INFO ][http ] [Rom the Spaceknight]
bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address
{inet[/10.0.0.108:9200]}
[2013-03-22 07:45:14,648][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: started

and status check gives the following:

curl http://10.0.0.108:9200/_status
{"error":"ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state
not recovered / initialized];[SERVICE_UNAVAILABLE/2/no master];]","status":
503}[ec2-user@ip-10-0-0-108 elasticsearch-0.20.5]

I'm experimenting with fresh instance of Amazon Linux AMI
[amzn-ami-pv-2012.09.1.x86_64-ebs (ami-54cf5c3d)]. Security groups allow
all outbound traffic and 9300-9400 ports for inbound TCP and UDP
connections.

Are there any special networking/VPC configurations required to make
ElasticSearch start correctly?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

We've resolved the issue by setting up NAT instance and configuring our VPC
properly.

Sorry for bothering you.

  • Ivan Yatskevich

On Friday, March 22, 2013 11:13:38 AM UTC+3, Ivan Yatskevich wrote:

Hi there,

We are trying to setup Elasticsearch cluster on AWS, but unfortunately
private instances inside VPC (without Elastic IPs) are not able to start
correctly.
Here is our *elasticsearch.yml:

**
discovery.type: ec2
discovery.ec2.ping_timeout: 10s
discovery.zen.ping.multicast.enabled: false

Start log:
*
[2013-03-22 07:44:36,833][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: initializing ...
[2013-03-22 07:44:36,874][INFO ][plugins ] [Rom the
Spaceknight] loaded [cloud-aws], sites
[2013-03-22
07:44:40,756][DEBUG][discovery.zen.ping.unicast] [Rom the Spaceknight]
using initial hosts , with concurrent_connects [10]
[2013-03-22
07:44:40,758][DEBUG][discovery.ec2 ] [Rom the Spaceknight]
using ping.timeout [10s], master_election.filter_client [true],
master_election.filter_data [false]
[2013-03-22 07:44:40,762][DEBUG][discovery.zen.elect ] [Rom the
Spaceknight] using minimum_master_nodes [-1]
[2013-03-22
07:44:40,763][DEBUG][discovery.zen.fd ] [Rom the Spaceknight]
[master] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2013-03-22
07:44:40,768][DEBUG][discovery.zen.fd ] [Rom the Spaceknight]
[node ] uses ping_interval [1s], ping_timeout [30s], ping_retries [3]
[2013-03-22
07:44:41,436][DEBUG][discovery.ec2 ] [Rom the Spaceknight]
using host_type [PRIVATE_IP], tags [{}], groups [] with any_group
[true], availability_zones []
[2013-03-22 07:44:44,107][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: initialized
[2013-03-22 07:44:44,107][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: starting ...
[2013-03-22
07:44:44,557][INFO ][transport ] [Rom the Spaceknight]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address
{inet[/10.0.0.108:9300]}
[2013-03-22
07:44:44,639][TRACE][discovery ] [Rom the Spaceknight]
waiting for 30s for the initial state to be set by the discovery
[2013-03-22
07:45:14,639][WARN ][discovery ] [Rom the Spaceknight]
waited for 30s and no initial state was set by the discovery
[2013-03-22 07:45:14,640][INFO ][discovery ] [Rom the
Spaceknight] ec2_vpc_private/JIIcSwi2SoKobfzXTeJCjg
[2013-03-22
07:45:14,647][INFO ][http ] [Rom the Spaceknight]
bound_address {inet[/0:0:0:0:0:0:0:0:9200]}, publish_address
{inet[/10.0.0.108:9200]}
[2013-03-22 07:45:14,648][INFO ][node ] [Rom the
Spaceknight] {0.20.5}[1595]: started

and status check gives the following:

curl http://10.0.0.108:9200/_status
{"error":"ClusterBlockException[blocked by: [SERVICE_UNAVAILABLE/1/state
not recovered / initialized];[SERVICE_UNAVAILABLE/2/no master];]","status"
:503}[ec2-user@ip-10-0-0-108 elasticsearch-0.20.5]

I'm experimenting with fresh instance of Amazon Linux AMI
[amzn-ami-pv-2012.09.1.x86_64-ebs (ami-54cf5c3d)]. Security groups allow
all outbound traffic and 9300-9400 ports for inbound TCP and UDP
connections.

Are there any special networking/VPC configurations required to make
Elasticsearch start correctly?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.