Ec2 & public ip

Karussell1 · September 26, 2011, 3:43pm

Hi,

I'm currently trying out Elasticsearch on an ec2 machine and
connecting to it via JSON or elasticsearch-head works using the public
IP. I simply want to make it insecure and open Elasticsearch to
everyone.

But using the transport client does not work - it waits some seconds
and then reports that there is no node available.
How can I tell elasticsearch to use the public IP adress as
elasticsearch says:
[2011-09-26 15:32:56,469][INFO ][transport ] [Starhawk]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
THE_PRIVATE_IP:9300]}

I tried this in the config:

network.publish_host.ec2: publicIpv4

but it does not work. Also I do not understand why the ES-HEAD works
over 9200 and the transport client not, although both are binding to
the private adress (both ports + http are open). Can someone explain
this to a network noob ?

Regards,
Peter.

Paul_Loy · September 26, 2011, 3:51pm

On Mon, Sep 26, 2011 at 4:43 PM, Karussell tableyourtime@googlemail.comwrote:

Hi,

I'm currently trying out Elasticsearch on an ec2 machine and
connecting to it via JSON or elasticsearch-head works using the public
IP. I simply want to make it insecure and open Elasticsearch to
everyone.

But using the transport client does not work - it waits some seconds
and then reports that there is no node available.
How can I tell elasticsearch to use the public IP adress as
elasticsearch says:
[2011-09-26 15:32:56,469][INFO ][transport ] [Starhawk]
bound_address {inet[/0:0:0:0:0:0:0:0:9300]}, publish_address {inet[/
THE_PRIVATE_IP:9300]}

I tried this in the config:

network.publish_host.ec2: publicIpv4

but it does not work. Also I do not understand why the ES-HEAD works
over 9200 and the transport client not, although both are binding to
the private adress (both ports + http are open). Can someone explain
this to a network noob ?

Regards,
Peter.

I think the config is slightly wrong, try this:

network.publish_host: ec2:publicIpv4

Paul Loy
paul@keteracel.com
http://uk.linkedin.com/in/paulloy

Karussell1 · September 26, 2011, 5:03pm

Cool, thanks Paul! now ES reports the public ip:

transport] [Razor Fist] bound_address {inet[/0:0:0:0:0:0:0:0:9300]},
publish_address {inet[/PUBLIC_IP:9300]}

but the transport client connection is not working.Not sure if this is
related but ES still reports private IP adresses at several points in
the logs:

gist.github.com

https://gist.github.com/karussell/1242740

gistfile1.txt


[2011-09-26 16:49:58,710][TRACE][discovery                ] [Razor Fist] waiting for 30s for the initial state to be set by the discovery
[2011-09-26 16:49:59,800][TRACE][discovery.ec2            ] [Razor Fist] building dynamic unicast discovery nodes...
[2011-09-26 16:49:59,821][TRACE][discovery.ec2            ] [Razor Fist] adding i-5a06613a, address PRIVATE_IP, transport_address inet[/PRIVATE_IP:9300]
[2011-09-26 16:49:59,821][DEBUG][discovery.ec2            ] [Razor Fist] using dynamic discovery nodes [[#cloud-i-5a06613a-0][inet[/PRIVATE_IP:9300]]]
[2011-09-26 16:49:59,872][TRACE][discovery.zen.ping.unicast] [Razor Fist] [1] connecting to [#cloud-i-5a06613a-0][inet[/PRIVATE_IP:9300]]
[2011-09-26 16:49:59,966][TRACE][discovery.zen.ping.unicast] [Razor Fist] [1] received response from [#cloud-i-5a06613a-0][inet[/PRIVATE_IP:9300]]: [ping_response{target [[Razor Fist][Otq02_OVRamD0VxK6f71cg][inet[/PUBLIC_IP:9300]]], master [null], cluster_name[elasticsearch]}, ping_response{target [[Razor Fist][Otq02_OVRamD0VxK6f71cg][inet[/PUBLIC_IP:9300]]], master [null], cluster_name[elasticsearch]}]
[2011-09-26 16:50:01,382][TRACE][discovery.ec2            ] [Razor Fist] building dynamic unicast discovery nodes...
[2011-09-26 16:50:01,386][TRACE][discovery.ec2            ] [Razor Fist] adding i-5a06613a, address PRIVATE_IP, transport_address inet[/PRIVATE_IP:9300]
[2011-09-26 16:50:01,386][DEBUG][discovery.ec2            ] [Razor Fist] using dynamic discovery nodes [[#cloud-i-5a06613a-0][inet[/PRIVATE_IP:9300]]]

This file has been truncated. show original

Karussell1 · September 26, 2011, 5:08pm

Do i need to configure the TransportClient (which is not on an ec2
machine) in a different way than the data node?

On 26 Sep., 19:03, Karussell tableyourt...@googlemail.com wrote:

Cool, thanks Paul! now ES reports the public ip:

transport] [Razor Fist] bound_address {inet[/0:0:0:0:0:0:0:0:9300]},
publish_address {inet[/PUBLIC_IP:9300]}

but the transport client connection is not working.Not sure if this is
related but ES still reports private IP adresses at several points in
the logs:

elasticsearch on ec2 ip problems · GitHub

kimchy · September 26, 2011, 5:27pm

The log that you see means that when a nodes starts up, and tries to check
for the list of nodes to connect to, it goes to the amazon API and fetches
the list of machines. By default, it will use the private IP to try and
connect to the node. You don't really need to change, but you can, by
setting discovery.ec2.host_type to public_ip.

The instance started binds to "any host" address. The fact that you can
connect to it using elasticsearch-head means that it works host wise. Are
you sure you opened the 9300 port in the security group or other firewall
settings?

On Mon, Sep 26, 2011 at 8:08 PM, Karussell tableyourtime@googlemail.comwrote:

Do i need to configure the TransportClient (which is not on an ec2
machine) in a different way than the data node?

On 26 Sep., 19:03, Karussell tableyourt...@googlemail.com wrote:

Cool, thanks Paul! now ES reports the public ip:

transport] [Razor Fist] bound_address {inet[/0:0:0:0:0:0:0:0:9300]},
publish_address {inet[/PUBLIC_IP:9300]}

but the transport client connection is not working.Not sure if this is
related but ES still reports private IP adresses at several points in
the logs:

elasticsearch on ec2 ip problems · GitHub

Karussell1 · September 27, 2011, 4:27pm

Hello Shay,

thanks for your answer!

I have re-enabled the port and but it is the same problem.
What I cannot understand is why I cannot even make it working when
using a ssh port forwarding:

ssh -l user -L 9999:localhost:9300 PUBLIC_IP

then using 9999 as port in the transport client (which btw does not
use the aws cloud settings)

I'll investigate it further ...

Regards,
Peter.

On 26 Sep., 19:27, Shay Banon kim...@gmail.com wrote:

The log that you see means that when a nodes starts up, and tries to check
for the list of nodes to connect to, it goes to the amazon API and fetches
the list of machines. By default, it will use the private IP to try and
connect to the node. You don't really need to change, but you can, by
setting discovery.ec2.host_type to public_ip.

The instance started binds to "any host" address. The fact that you can
connect to it using elasticsearch-head means that it works host wise. Are
you sure you opened the 9300 port in the security group or other firewall
settings?

On Mon, Sep 26, 2011 at 8:08 PM, Karussell tableyourt...@googlemail.comwrote:

Do i need to configure the TransportClient (which is not on an ec2
machine) in a different way than the data node?

On 26 Sep., 19:03, Karussell tableyourt...@googlemail.com wrote:

Cool, thanks Paul! now ES reports the public ip:

transport] [Razor Fist] bound_address {inet[/0:0:0:0:0:0:0:0:9300]},
publish_address {inet[/PUBLIC_IP:9300]}

but the transport client connection is not working.Not sure if this is
related but ES still reports private IP adresses at several points in
the logs:

elasticsearch on ec2 ip problems · GitHub

Karussell1 · September 29, 2011, 12:57pm

After creating a new ec2 machine it now works ...

On 27 Sep., 18:27, Karussell tableyourt...@googlemail.com wrote:

Hello Shay,

thanks for your answer!

I have re-enabled the port and but it is the same problem.
What I cannot understand is why I cannot even make it working when
using a ssh port forwarding:

ssh -l user -L 9999:localhost:9300 PUBLIC_IP

then using 9999 as port in the transport client (which btw does not
use the aws cloud settings)

I'll investigate it further ...

Regards,
Peter.

On 26 Sep., 19:27, Shay Banon kim...@gmail.com wrote:

The log that you see means that when a nodes starts up, and tries to check
for the list of nodes to connect to, it goes to the amazon API and fetches
the list of machines. By default, it will use the private IP to try and
connect to the node. You don't really need to change, but you can, by
setting discovery.ec2.host_type to public_ip.

The instance started binds to "any host" address. The fact that you can
connect to it using elasticsearch-head means that it works host wise. Are
you sure you opened the 9300 port in the security group or other firewall
settings?

On Mon, Sep 26, 2011 at 8:08 PM, Karussell tableyourt...@googlemail.comwrote:

Do i need to configure the TransportClient (which is not on anec2
machine) in a different way than the data node?

On 26 Sep., 19:03, Karussell tableyourt...@googlemail.com wrote:

Cool, thanks Paul! now ES reports the public ip:

transport] [Razor Fist] bound_address {inet[/0:0:0:0:0:0:0:0:9300]},
publish_address {inet[/PUBLIC_IP:9300]}

but the transport client connection is not working.Not sure if this is
related but ES still reports private IP adresses at several points in
the logs:

elasticsearch on ec2 ip problems · GitHub