We have a use case where a small ES cluster is in OpenStack and needs to be available to OS's external network. I currently have this network config structure in elasticsearch.yml:
where 192.168.29.0/24 is the OpenStack project internal network and the other address is its assigned floating IP (each of the 5 nodes has its own floating IP). Reasoning here is so that inter-node communications on port 9300 don't have to traverse networks and end up with higher latency via the OpenStack network driver, and external client discovery tools like the Sniffer functionality of the official elasticsearch-py module will see the external IPs they require to function properly.
I am finding that this config causes the host field to reflect the internal network IP rather than what is published via network.publish_host (GET /_nodes/_all/clear exerpt):
transport.publish_host respects its default unspecified value of what is set for network.publish_host per the network settings module docs.
So, am I going about this completely wrong? Am I stuck between higher transport latency and broken discovery tools? Or is there a better way to divorce the two ports from the same network? Is this a bug?
transport_address exposes the published address for the transport and is the address used by other nodes to talk to this node. When you set transport.publish_host to 192.168.29.16, the transport address will properly reflect that.
network.publish_host is used as a fallback for both transport.publish_host and http.publish_host if they're not defined.
Your reply makes me realize I'm not articulating my question properly.
A sniffing tool, such as elasticsearch-py searches for field host in the output for endpoint GET /_nodes/_all/clear. In my case, any sniffer will be on the external network 10.104.1.x (or beyond) and needs to see host with a 10.104.1.x address.
The issue I'm having is that when transport.publish_host is defined, it overrides the setting of network.publish_host. Following my output from GET /_nodes/_all/clear above, the desired output should be:
This way, transport is still occurring internal to the OpenStack project, and sniffing tools can communicate with the stack because the returned IPs are routable.
Does that make more sense? So the question is, how do I get this desired output? Is this a bug or am I doing something functionally wrong?
OK, I see what you're getting at. The output of GET /_nodes/_all/clear won't give you what you need, however. The field host in the given output will always display the transport publish address that is used for inter-node communication. If you have a sniffer that goes by this field, it won't give you the desired behavior.
The transport protocol is used by Elasticsearch for inter-node communication. Clients use the HTTP protocol (with the exception of the Java transport client, which also uses the transport protocol). The settings transport.bind_host and transport.publish_host only apply to the transport protocol. If I understand you correctly, you're interested in connecting to the cluster using http. Here, you can configure a separate http.bind_host and http.publish_host. Using the configuration you gave in the first post, http.bind_host will point to 192.168.29.16 and http.publish_host to 10.104.1.170 as these fall back to network.bind_host and network.publish_host if not explicitly defined.
The http publish host is also exposed in the node stats API, if you query for GET /_nodes/http. I'm not too familiar with the elasticsearch-py sniffer but maybe it can be configured to use the value exposed in the http.publish_address field?
Sorry for the late reply. Thanks for reminding me of the http module; I haven't yet tested this but will report back when I am able to effect the changes.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.