How to Specifying ES option via Shell environment variables for discovery.zen.ping.unicast.hosts

Elastic Stack Elasticsearch
1

Hello. I'm using ES 2.3.2

I've decided to use Shell Env. Variables in elasticsearch.yml because I want to maintain ONLY 1 elasticsearch.yml file even though I have many type of nodes in many clusters.

Most of configs are easy to use shell variables. But I turned out that discovery.zen.ping.unicast.hosts is weird.

original elasticsearch.yml
discovery.zen.ping.unicast.hosts: ["host1", "host2", "host3"]

First attempt

I've changed like this:

discovery.zen.ping.unicast.hosts: [${UNICAST_HOSTS}]

And in the shell

export UNICAST_HOSTS='"host1", "host2", "host3"'

But I got an error (Maybe Invalid Placeholder? Sorry. I can't remember the exact error message).

Second attemp

  • elasticsearch.yml: discovery.zen.ping.unicast.hosts: ${UNICAST_HOSTS}
  • shell: export UNICAST_HOSTS='["host1", "host2", "host3"']

I got an error as well.

Final Solution

After several trials and errors, I've found that following configuration worked!

  • elasticsearch.yml: discovery.zen.ping.unicast.hosts: ${UNICAST_HOSTS}
  • shell: export UNICAST_HOSTS="host1, host2, host3"

It works fine. But where have [, ] gone? Is above configuration is correct? It would be happy if someone tell me "YES" or "NO". I'm not certain because I can't find any authorized documentation.

2

Hi @Brad_Jungsu_Heo

As far as I can see, the configuration is correct. You can add the following line to config/logging.yml:

discovery.zen.ping.unicast: DEBUG

This enables debugging logging in the relevant module. On startup you will then see the following line:

[2016-07-27 13:46:52,028][DEBUG][discovery.zen.ping.unicast] [-96ewfK] using initial hosts [host1, host2, host3], with concurrent_connects [10]

If this would not get parsed as a list and instead resolved as one host, you should see exceptions traces like this one on startup (which you obviously don't see):

Error injecting constructor, java.lang.IllegalArgumentException: Failed to resolve address for [host1, host2, host3]
  at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.<init>(Unknown Source)
  while locating org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing
  while locating org.elasticsearch.discovery.zen.ping.ZenPing annotated with @org.elasticsearch.common.inject.multibindings.Element(setName=,uniqueId=1)
  at _unknown_
  while locating java.util.Set<org.elasticsearch.discovery.zen.ping.ZenPing>
    for parameter 1 at org.elasticsearch.discovery.zen.ping.ZenPingService.<init>(Unknown Source)
  while locating org.elasticsearch.discovery.zen.ping.ZenPingService
    for parameter 5 at org.elasticsearch.discovery.zen.ZenDiscovery.<init>(Unknown Source)
  while locating org.elasticsearch.discovery.zen.ZenDiscovery
  while locating org.elasticsearch.discovery.Discovery
Caused by: java.lang.IllegalArgumentException: Failed to resolve address for [host1, host2, host3]
    at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.<init>(UnicastZenPing.java:171)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at [...]
Caused by: java.net.UnknownHostException: host1, host2, host3
    at java.net.InetAddress.getAllByName0(InetAddress.java:1280)
    at java.net.InetAddress.getAllByName(InetAddress.java:1192)
    at java.net.InetAddress.getAllByName(InetAddress.java:1126)
    at org.elasticsearch.transport.TcpTransport.parse(TcpTransport.java:748)
    at org.elasticsearch.transport.TcpTransport.addressesFromString(TcpTransport.java:703)
    at org.elasticsearch.transport.TransportService.addressesFromString(TransportService.java:565)
    at org.elasticsearch.discovery.zen.ping.unicast.UnicastZenPing.<init>(UnicastZenPing.java:165)
    ... 60 more

Once more: This was just a hypothetical example what would happen if Elasticsearch were not able to resolve this correctly.

Daniel

3

Hi @danielmitterdorfer

Thank you so much!