Discovery on EC2 - unicast, separate VPCs, public IPs

Elastic Stack Elasticsearch
1

Hi,
I have the following simple EC2 topology:

  • a VPC with my entire cluster, running in a public subnet
  • a new slave in another VPC (also a public subnet)
  • I'm using unicast - the slave has the following config:
    discovery.zen.ping.multicast.enabled: false
    discovery.zen.ping.unicast.hosts: ["master_elastic_ip:9300"]
    So - the slave points to the public IP of the master - not the private one.

However - this new slave tries to connect to the master on the private IP
instead of the public one - and I'm getting:
org.elasticsearch.common.netty.channel.ConnectTimeoutException: connection
timed out: /172.61.51.253:9300
Where 172.61.51.253 is the private IP.
Not sure what that is - do I need to configure anything on the slave to
make sure it uses the public IP to reach the master?
Thanks,
Eugen.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

Are you referring to the master server by fqdn or IP? If fqdn, don't
forget about ec2's split horizon (though I don't think it should be
resolvable across 2 separate VPCs...
Can you open a socket from host 1 to host 2 manually (with nc or telnet) on
TCP/9300?
On 08/02/2015 9:43 am, "Eugen Paraschiv" hanriseldon@gmail.com wrote:

Hi,
I have the following simple EC2 topology:

  • a VPC with my entire cluster, running in a public subnet
  • a new slave in another VPC (also a public subnet)
  • I'm using unicast - the slave has the following config:
    discovery.zen.ping.multicast.enabled: false
    discovery.zen.ping.unicast.hosts: ["master_elastic_ip:9300"]
    So - the slave points to the public IP of the master - not the private
    one.

However - this new slave tries to connect to the master on the private IP
instead of the public one - and I'm getting:
org.elasticsearch.common.netty.channel.ConnectTimeoutException: connection
timed out: /172.61.51.253:9300
Where 172.61.51.253 is the private IP.
Not sure what that is - do I need to configure anything on the slave to
make sure it uses the public IP to reach the master?
Thanks,
Eugen.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACj2-4JqqAynnkvJ85JW3EphiM22_ONx1xHFH96-SrjZ2UKb6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

DNS queries inside vpc are resolved to the internal IP by aws servers.

One simple way is to use elastic IP for this computer and then point A
register to that IP.
El 07/02/2015 23:43, "Eugen Paraschiv" hanriseldon@gmail.com escribió:

Hi,
I have the following simple EC2 topology:

  • a VPC with my entire cluster, running in a public subnet
  • a new slave in another VPC (also a public subnet)
  • I'm using unicast - the slave has the following config:
    discovery.zen.ping.multicast.enabled: false
    discovery.zen.ping.unicast.hosts: ["master_elastic_ip:9300"]
    So - the slave points to the public IP of the master - not the private
    one.

However - this new slave tries to connect to the master on the private IP
instead of the public one - and I'm getting:
org.elasticsearch.common.netty.channel.ConnectTimeoutException: connection
timed out: /172.61.51.253:9300
Where 172.61.51.253 is the private IP.
Not sure what that is - do I need to configure anything on the slave to
make sure it uses the public IP to reach the master?
Thanks,
Eugen.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BjeyjOO6tU3qEsyf3xcD7QgZD_e-CM_g-oM9%2BRt%2B3LpBWPwUg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

4

Sure...the interesting point in the OP is the fact both servers are in
different VPCs - not sure if it should be possible to resolve across vpcs
...
On 08/02/2015 7:04 pm, "Ivan G" igonzalezvaliente@gmail.com wrote:

DNS queries inside vpc are resolved to the internal IP by aws servers.

One simple way is to use elastic IP for this computer and then point A
register to that IP.
El 07/02/2015 23:43, "Eugen Paraschiv" hanriseldon@gmail.com escribió:

Hi,
I have the following simple EC2 topology:

  • a VPC with my entire cluster, running in a public subnet
  • a new slave in another VPC (also a public subnet)
  • I'm using unicast - the slave has the following config:
    discovery.zen.ping.multicast.enabled: false
    discovery.zen.ping.unicast.hosts: ["master_elastic_ip:9300"]
    So - the slave points to the public IP of the master - not the private
    one.

However - this new slave tries to connect to the master on the private IP
instead of the public one - and I'm getting:
org.elasticsearch.common.netty.channel.ConnectTimeoutException:
connection timed out: /172.61.51.253:9300
Where 172.61.51.253 is the private IP.
Not sure what that is - do I need to configure anything on the slave to
make sure it uses the public IP to reach the master?
Thanks,
Eugen.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BjeyjOO6tU3qEsyf3xcD7QgZD_e-CM_g-oM9%2BRt%2B3LpBWPwUg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2BjeyjOO6tU3qEsyf3xcD7QgZD_e-CM_g-oM9%2BRt%2B3LpBWPwUg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACj2-4LNYv%2BrZZcZOt8rJTU6QcA1qAVdHNJzCutFzWwx%2B32Eew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

It will work for sure, I was using this approach on some use cases

The point is that if this server is not going to be running usually, aws
will continue charging you for the elastic ip even if no instance is up in
that IP, but for the cost it is, I think it represents, the elastic ip is
the best option.
That said you can write your own script that register as A record the
current public IP of the instance and avoid use elastic ip's.

--

Iván González Valiente

Senior Systems Developer

Tel. +34 675 962 588

2015-02-08 9:20 GMT+01:00 Norberto Meijome numard@gmail.com:

Sure...the interesting point in the OP is the fact both servers are in
different VPCs - not sure if it should be possible to resolve across vpcs
...
On 08/02/2015 7:04 pm, "Ivan G" igonzalezvaliente@gmail.com wrote:

DNS queries inside vpc are resolved to the internal IP by aws servers.

One simple way is to use elastic IP for this computer and then point A
register to that IP.
El 07/02/2015 23:43, "Eugen Paraschiv" hanriseldon@gmail.com escribió:

Hi,
I have the following simple EC2 topology:

  • a VPC with my entire cluster, running in a public subnet
  • a new slave in another VPC (also a public subnet)
  • I'm using unicast - the slave has the following config:
    discovery.zen.ping.multicast.enabled: false
    discovery.zen.ping.unicast.hosts: ["master_elastic_ip:9300"]
    So - the slave points to the public IP of the master - not the private
    one.

However - this new slave tries to connect to the master on the private
IP instead of the public one - and I'm getting:
org.elasticsearch.common.netty.channel.ConnectTimeoutException:
connection timed out: /172.61.51.253:9300
Where 172.61.51.253 is the private IP.
Not sure what that is - do I need to configure anything on the slave to
make sure it uses the public IP to reach the master?
Thanks,
Eugen.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/026f6d30-d496-4905-a5f9-80c6be82669b%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CA%2BjeyjOO6tU3qEsyf3xcD7QgZD_e-CM_g-oM9%2BRt%2B3LpBWPwUg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CA%2BjeyjOO6tU3qEsyf3xcD7QgZD_e-CM_g-oM9%2BRt%2B3LpBWPwUg%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CACj2-4LNYv%2BrZZcZOt8rJTU6QcA1qAVdHNJzCutFzWwx%2B32Eew%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CACj2-4LNYv%2BrZZcZOt8rJTU6QcA1qAVdHNJzCutFzWwx%2B32Eew%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CA%2BjeyjNZn2Auv60V9L-gL%2BzJcJDB4%2Bwb4B%2BTkqX0oXNOADaoew%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.