Network.publish_host does not seem to do what it says it does

Elastic Stack Elasticsearch
1

I'm trying to bind the API to 127.0.0.1 and use the apache reverse proxy
settings with kibana to talk to ES. This works as I expected.

However, I need to bind the inter-cluster communication to a real IP
address, supposedly with network.publish_host, but this does not work.
Both bind to 127.0.0.1

Running from the latest deb package, on ubuntu 12.04

from /etc/elasticsearch/elasticsearch.yaml

network.bind_host: 127.0.0.1
network.publish_host: my.redacted.ip.address

netstat -an | grep 9300
tcp 0 0 127.0.0.1:9300 0.0.0.0:* LISTEN

netstat -an | grep 9200
tcp 0 0 127.0.0.1:9200 0.0.0.0:* LISTEN

this is a major deal as I do not want to bind the api directly to a
routable interface.

from the docs:

The network.bind_host setting allows to control the host different network
components will bind on. By default, the bind host will be anyLocalAddress
(typically 0.0.0.0 or ::0).

The network.publish_host setting allows to control the host the node will
publish itself within the cluster so other nodes will be able to connect to
it. Of course, this can’t be the anyLocalAddress, and by default, it will
be the first non loopback address (if possible), or the local address.

The network.host setting is a simple setting to automatically set both
network.bind_host and network.publish_host to the same host value.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

As far as I am aware you can't split the two, they both need to be on the
same interface/IP.
I don't know if this is a bug or if the docs are incorrect.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 17 April 2014 08:28, OJ LaBoeuf orangepeelbeef@gmail.com wrote:

I'm trying to bind the API to 127.0.0.1 and use the apache reverse proxy
settings with kibana to talk to ES. This works as I expected.

However, I need to bind the inter-cluster communication to a real IP
address, supposedly with network.publish_host, but this does not work.
Both bind to 127.0.0.1

Running from the latest deb package, on ubuntu 12.04

from /etc/elasticsearch/elasticsearch.yaml

network.bind_host: 127.0.0.1
network.publish_host: my.redacted.ip.address

netstat -an | grep 9300
tcp 0 0 127.0.0.1:9300 0.0.0.0:*
LISTEN

netstat -an | grep 9200
tcp 0 0 127.0.0.1:9200 0.0.0.0:*
LISTEN

this is a major deal as I do not want to bind the api directly to a
routable interface.

from the docs:

The network.bind_host setting allows to control the host different
network components will bind on. By default, the bind host will be
anyLocalAddress (typically 0.0.0.0 or ::0).

The network.publish_host setting allows to control the host the node will
publish itself within the cluster so other nodes will be able to connect to
it. Of course, this can't be the anyLocalAddress, and by default, it will
be the first non loopback address (if possible), or the local address.

The network.host setting is a simple setting to automatically set both
network.bind_host and network.publish_host to the same host value.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZP-_h720RhceosOs%2B1mjKByxYa1gSP_hv%3DzeYfSx4FKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

3

That doesn't make a lot of sense then. Why would you be able to set the
publish host to something different if they both have to bind to the same
interface? I'm not understanding what the purpose of these configuration
bits is. I was also told in the IRC room that this was how you separated
them a while back...

I can't really believe that nobody else has run into this? Why would you
want to bind your zero security direct access api to a routable layer 3
address if you don't have to?

Kibana proxypass config is also set to use 127.0.0.1 as its bind, is it
then expected that there is no cluster capability whatsoever?

The only alternative I can think of is to bind it on 0.0.0.0 and then rely
on host based firewall to only allow connection via 127.0.0.1 for 9200 and
l3 ip for 9300, but this seems like something the app should be able to do.

On Wednesday, April 16, 2014 4:20:03 PM UTC-7, Mark Walkom wrote:

As far as I am aware you can't split the two, they both need to be on the
same interface/IP.
I don't know if this is a bug or if the docs are incorrect.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com <javascript:>
web: www.campaignmonitor.com

On 17 April 2014 08:28, OJ LaBoeuf <orangep...@gmail.com <javascript:>>wrote:

I'm trying to bind the API to 127.0.0.1 and use the apache reverse proxy
settings with kibana to talk to ES. This works as I expected.

However, I need to bind the inter-cluster communication to a real IP
address, supposedly with network.publish_host, but this does not work.
Both bind to 127.0.0.1

Running from the latest deb package, on ubuntu 12.04

from /etc/elasticsearch/elasticsearch.yaml

network.bind_host: 127.0.0.1
network.publish_host: my.redacted.ip.address

netstat -an | grep 9300
tcp 0 0 127.0.0.1:9300 0.0.0.0:*
LISTEN

netstat -an | grep 9200
tcp 0 0 127.0.0.1:9200 0.0.0.0:*
LISTEN

this is a major deal as I do not want to bind the api directly to a
routable interface.

from the docs:

The network.bind_host setting allows to control the host different
network components will bind on. By default, the bind host will be
anyLocalAddress (typically 0.0.0.0 or ::0).

The network.publish_host setting allows to control the host the node
will publish itself within the cluster so other nodes will be able to
connect to it. Of course, this can’t be the anyLocalAddress, and by
default, it will be the first non loopback address (if possible), or the
local address.

The network.host setting is a simple setting to automatically set both
network.bind_host and network.publish_host to the same host value.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4a10c90b-4069-46bf-ad6b-fa41cadaeea1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

The notes I recorded from the ES training I did a while back are my source,
but I may be wrong.
Hopefully someone else can clarify or confirm.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 17 April 2014 16:23, OJ LaBoeuf orangepeelbeef@gmail.com wrote:

That doesn't make a lot of sense then. Why would you be able to set the
publish host to something different if they both have to bind to the same
interface? I'm not understanding what the purpose of these configuration
bits is. I was also told in the IRC room that this was how you separated
them a while back...

I can't really believe that nobody else has run into this? Why would you
want to bind your zero security direct access api to a routable layer 3
address if you don't have to?

Kibana proxypass config is also set to use 127.0.0.1 as its bind, is it
then expected that there is no cluster capability whatsoever?

The only alternative I can think of is to bind it on 0.0.0.0 and then
rely on host based firewall to only allow connection via 127.0.0.1 for 9200
and l3 ip for 9300, but this seems like something the app should be able
to do.

On Wednesday, April 16, 2014 4:20:03 PM UTC-7, Mark Walkom wrote:

As far as I am aware you can't split the two, they both need to be on the
same interface/IP.
I don't know if this is a bug or if the docs are incorrect.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com

On 17 April 2014 08:28, OJ LaBoeuf orangep...@gmail.com wrote:

I'm trying to bind the API to 127.0.0.1 and use the apache reverse
proxy settings with kibana to talk to ES. This works as I expected.

However, I need to bind the inter-cluster communication to a real IP
address, supposedly with network.publish_host, but this does not work.
Both bind to 127.0.0.1

Running from the latest deb package, on ubuntu 12.04

from /etc/elasticsearch/elasticsearch.yaml

network.bind_host: 127.0.0.1
network.publish_host: my.redacted.ip.address

netstat -an | grep 9300
tcp 0 0 127.0.0.1:9300 0.0.0.0:*
LISTEN

netstat -an | grep 9200
tcp 0 0 127.0.0.1:9200 0.0.0.0:*
LISTEN

this is a major deal as I do not want to bind the api directly to a
routable interface.

from the docs:

The network.bind_host setting allows to control the host different
network components will bind on. By default, the bind host will be
anyLocalAddress (typically 0.0.0.0 or ::0).

The network.publish_host setting allows to control the host the node
will publish itself within the cluster so other nodes will be able to
connect to it. Of course, this can't be the anyLocalAddress, and by
default, it will be the first non loopback address (if possible), or the
local address.

The network.host setting is a simple setting to automatically set both
network.bind_host and network.publish_host to the same host value.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%
40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4a10c90b-4069-46bf-ad6b-fa41cadaeea1%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/4a10c90b-4069-46bf-ad6b-fa41cadaeea1%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bzaMWgajKcYQS90E1Wn0JqNgJi687HodYiphOO-F3Fbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

5

Due to limitations of TCP/IP server socket bind, you can do the following:

  • just use network.host to configure an IP address. This is the most common
    case.

  • use network.bind_host to a single internal IP address and
    network.publish_host to a single external IP address on the same(!) network
    interface (also known as virtual IP address). ES uses the "bind all"
    address as a fall back if the bind fails.

  • use the "bind all" address = 0.0.0.0 (or IPv6 ::1) for network.host if
    you want to use different (=all) network interfaces you want ES to listen on

The name "publish_host" is a bit unfortunate. This is not a second IP
address on another interface.

Jörg

On Thu, Apr 17, 2014 at 12:28 AM, OJ LaBoeuf orangepeelbeef@gmail.comwrote:

I'm trying to bind the API to 127.0.0.1 and use the apache reverse proxy
settings with kibana to talk to ES. This works as I expected.

However, I need to bind the inter-cluster communication to a real IP
address, supposedly with network.publish_host, but this does not work.
Both bind to 127.0.0.1

Running from the latest deb package, on ubuntu 12.04

from /etc/elasticsearch/elasticsearch.yaml

network.bind_host: 127.0.0.1
network.publish_host: my.redacted.ip.address

netstat -an | grep 9300
tcp 0 0 127.0.0.1:9300 0.0.0.0:*
LISTEN

netstat -an | grep 9200
tcp 0 0 127.0.0.1:9200 0.0.0.0:*
LISTEN

this is a major deal as I do not want to bind the api directly to a
routable interface.

from the docs:

The network.bind_host setting allows to control the host different
network components will bind on. By default, the bind host will be
anyLocalAddress (typically 0.0.0.0 or ::0).

The network.publish_host setting allows to control the host the node will
publish itself within the cluster so other nodes will be able to connect to
it. Of course, this can’t be the anyLocalAddress, and by default, it will
be the first non loopback address (if possible), or the local address.

The network.host setting is a simple setting to automatically set both
network.bind_host and network.publish_host to the same host value.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/632b4ca3-6dad-4476-9b35-a6806b9fc783%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFnEHJHaNOSDua6fO8%3DtrqyTYp%2BbFP%2B-CabZ1EGqM61PA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

6

The doc of Elasticsearch about network.host should be updated in order to reflect your three options.

I've spent a lot of time searching for "bind_all" since Elasticsearch 2.0.

7

I've been trying this (as you suggested):

network.publish_host: _ec2:privateIpv4_
network.bind_host: 0.0.0.0

As I want to be able to curl localhost:9200 as well as curl http://mycluster.org:9200, and it appears that if you set bind_host to 0.0.0.0 independent of publish_host, it'll advertise its address as 127.0.0.1 and then you'll get this:

[2016-05-09 21:45:18,484][WARN ][discovery.zen.ping.unicast] [client_0] failed to send ping to [{#zen_unicast_1#}{127.0.0.1}{127.0.0.1:9300}]
SendRequestTransportException[[127.0.0.1:9300][internal:discovery/zen/unicast]]; nested: NodeNotConnectedException[[127.0.0.1:9300] Node not connected];

This doesn't really make any sense to me, I would expect publish_host to be the ip advertised to other elasticsearch instances, and bind_host only to affect how listen works.

Furthermore if you happen to configure your cluster this way, it seems totally stuck in this fashion!

1 Like