Discovery service as endpoint provider not recognised

nealder · August 10, 2023, 7:00am

Hi Everyone,
I have submitted a bug ticket on github, but I got redirected here. Here is the ticket.
In short I found that the 'discovery.zen.ping.unicast.host' could resolve IP addresses of nodes in my cluster via discovery service name, but 'discovery.seed_hosts' could only resolve them if the static IP of the discovery service were given.
The problem here:

Documentation lacks this information
Hard coding IP of discovery service could lead to problems as the cluster naturally scales and redistributes between nodes.

I find this a regression, a feature loss therefore a bug. What are you opinions?

DavidTurner · August 10, 2023, 7:32am

These docs specifically say that discovery.seed_hosts will do DNS lookups:

The host is either a host name to be resolved by DNS, an IPv4 address, or an IPv6 address. IPv6 addresses must be enclosed in square brackets. If a host name resolves via DNS to multiple addresses, Elasticsearch uses all of them. DNS lookups are subject to JVM DNS caching.

Moreover there really is no difference between the handling of discovery.zen.ping.unicast.host and discovery.seed_hosts, they both feed into the configuredHosts variable in the following code and from then on there is no distinction:

github.com

elastic/elasticsearch/blob/bcfbf000748b538884087623113682165abcb92d/server/src/main/java/org/elasticsearch/discovery/SettingsBasedSeedHostsProvider.java#L54-L67


      
          if (LEGACY_DISCOVERY_ZEN_PING_UNICAST_HOSTS_SETTING.exists(settings)) {
              if (DISCOVERY_SEED_HOSTS_SETTING.exists(settings)) {
                  throw new IllegalArgumentException(
                      "it is forbidden to set both ["
                          + DISCOVERY_SEED_HOSTS_SETTING.getKey()
                          + "] and ["
                          + LEGACY_DISCOVERY_ZEN_PING_UNICAST_HOSTS_SETTING.getKey()
                          + "]"
                  );
              }
              configuredHosts = LEGACY_DISCOVERY_ZEN_PING_UNICAST_HOSTS_SETTING.get(settings);
              // we only limit to 1 address, makes no sense to ping 100 ports
          } else if (DISCOVERY_SEED_HOSTS_SETTING.exists(settings)) {
              configuredHosts = DISCOVERY_SEED_HOSTS_SETTING.get(settings);

I think the difference in behaviour you're seeing is most likely environmental.

nealder · August 10, 2023, 9:10am

Hi David,
unfortunatelly we already eliminated all environmental changes. It is literally only the elasticsearch image version and this discovery setting as differences.
Elasticsearch 7.17.0 has a feature which allows to do this and 8.8.1 does not have it under this setting. I tried to dig into the java code, but I'm no expert in that.
Question: how can one handle a cluster restart, scaling, redistribution or restarting of nodes?

DavidTurner · August 10, 2023, 10:14am

Apart from the removal of the legacy setting I cannot find any difference in behaviour in ES between 7.17 and 8.x. They both do DNS name lookups just fine.

system · September 7, 2023, 10:15am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Using DNS to define master list in discovery.zen.ping.unicast.hosts Elasticsearch	4	1804	July 5, 2017
How to remove discovery.zen.ping.unicasts completely Elasticsearch	3	432	July 2, 2022
Discovery.zen.ping.unicast.hosts find the wrong ip Elasticsearch	3	753	July 20, 2017
Clarification around unicast host discovery w/ round-robin dns Elasticsearch	13	1560	July 5, 2017
Setting for option discovery.zen.ping.unicast.hosts in elasticsearch.yml Elasticsearch	12	1286	December 6, 2018

Discovery service as endpoint provider not recognised

Related topics