ES 6.5.2 Problem making zen unicast discovery to work when transport.tcp.port is set

ld_pvl · January 29, 2019, 7:58pm

Hi Team,

I have set my discovery.zen.ping.unicast.hosts: ["hostA", "hostB", "hostC"] without a port range which according to the docs I believe should scan a range of ports by default which looks like it does.

Now for one of my coordinator nodes that has the above discovery.zen.ping.unicast.hosts setting, I want to set a custom fixed transport port say 9500 so I set transport.tcp.port: 9500 but when the coordinator starts up it complains that it cannot find any master nodes.

Looking at the logs, it looks like for some reason the coordinator overrides the default zen unicast discovery port range with port 9500 which makes it unable to discovery anything.

I tried to play around and set transport.profiles.default.port: 9300-9400 cos according to the docs:

The list of hosts is set using the discovery.zen.ping.unicast.hosts static setting. This is either an array of hosts or a comma-delimited string. Each value should be in the form of host:port or host (where port defaults to the setting transport.profiles.default.port falling back to transport.tcp.port if not set).

So I thought it might work but then this seems to replace transport.tcp.port which makes my coordinator not bind to transport port 9500.

In summary, what I want my coordinator to have is a "transport_address" : "SOME_IP:9500" and still be able to discover things with discovery.zen.ping.unicast.hosts: ["hostA", "hostB", "hostC"] - is that possible?

Many thanks for your help,

DavidTurner · January 30, 2019, 8:56am

Elasticsearch will scan five ports on localhost if you do not set discovery.zen.ping.unicast.hosts but does not do any port scanning if you do set this setting. This is a convenience feature for quick setups on a single host, but in any realistic production cluster you should give a specific port to each master-eligible node and specify that port in the addresses given in the discovery.zen.ping.unicast.hosts setting if needed:

discovery.zen.ping.unicast.hosts: ["hostA", "hostB:9500", "hostC"]

You only need to be able to discover the master-eligible nodes, so you can bind master-ineligible nodes to port ranges if you want since their ports do not really matter.

The relevant code, if you're interested, is here:

github.com

elastic/elasticsearch/blob/022726011ce1d9e4748fd39293a3aeae6954b3ac/server/src/main/java/org/elasticsearch/discovery/zen/SettingsBasedHostsProvider.java#L50-L66


public static final int LIMIT_FOREIGN_PORTS_COUNT = 1;
public static final int LIMIT_LOCAL_PORTS_COUNT = 5;


private final List<String> configuredHosts;


private final int limitPortCounts;


public SettingsBasedHostsProvider(Settings settings, TransportService transportService) {
    if (DISCOVERY_ZEN_PING_UNICAST_HOSTS_SETTING.exists(settings)) {
        configuredHosts = DISCOVERY_ZEN_PING_UNICAST_HOSTS_SETTING.get(settings);
        // we only limit to 1 address, makes no sense to ping 100 ports
        limitPortCounts = LIMIT_FOREIGN_PORTS_COUNT;
    } else {
        // if unicast hosts are not specified, fill with simple defaults on the local machine
        configuredHosts = transportService.getLocalAddresses();
        limitPortCounts = LIMIT_LOCAL_PORTS_COUNT;
    }

ld_pvl · January 30, 2019, 10:08am

Thanks David, it is much clearer now.

system · February 27, 2019, 10:08am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.