Does Elasticsearch pick up OS environment variable automatically, or I have to reference it in elasticsearch.yml explicitly

[morningspace]

I was using "reindex.remote.whitelist" to configure whitelist hosts for my remote reindex. Previously, I define that as normal environment variable w/o touching elasticsearch.yml. This works perfect on my local machine. So, when you check elasticsearch.yml, no reindex.remote.whitelist there, and if you printenv in OS terminal, you will see the environment variable being listed, e.g. reindex.remote.whitelist=my-elasticsearch:9200.

However, recently I was deploying the same stuff on a different env, where I got the error: [my-elasticsearch:9200] not whitelisted in reindex.remote.whitelist. If I check the /_cluster/settings, I do see reindex.remote.whitelist is empty.

It makes me a bit confusing. Does it mean I have to define that in elasticsearch.yml at first with a placeholder for the environment variable to be replaced later... Just like this:

# In elasticsearch.yml
reindex.remote.whitelist: "${REINDEX_REMOTE_WHITELIST}"

Then export REINDEX_REMOTE_WHITELIST in OS terminal.

Anything I missed?

BTW: I was trying to read the source code to understand where the code picks up the settings... I can see the Settings object being passed into TransportReindexAction, but still have no idea where the Settings object comes from

[xavierfacq]

I would say no, it doesn't. But is you realy need that, may be you can pass the variable as a param of the JVM in jmv.options ?

[morningspace]

Thanks @xavierfacq... well, actually I'm wrapping elasticsearch into Docker container... In prod env, we use K8s, while on my local machine, I use Docker Compose to instrument the env vars. But I think either way they are the same essentially.

Interestingly, when I use docker-compose.yml locally, and define the env var like this:

  elasticsearch:
    ...
    environment:
      reindex.remote.whitelist: elasticsearch-old:9200
      REINDEX_REMOTE_WHITELIST: elasticsearch-new:9200

Then in elasticsearch.yml:

reindex.remote.whitelist: "${REINDEX_REMOTE_WHITELIST}"

If I comment the first line in environment section in docker-compose.yml, launch the container, then check the _cluster/settings?include_defaults=true, I can see the value of reindex.remote.whitelist is elasticsearch-new:9200;

If I comment the second line, the value will be elasticsearch-old:9200;

If I keep both uncommented, the value will be elasticsearch-old:9200! It looks to define reindex.remote.whitelist as env var will have the higher priority than the other one. Really curious about what's the magic happened behind...

P.S.
I'd say to define reindex.remote.whitelist as env var directly may not be a common way?... Actually people cannot export env var w/ dots like this in Bash. But I do see that works on my local... Just it doesn't work in prod env... which makes me confused... is that because of the OS difference? I'm using the official Docker image on local which is based on CentOS, while in prod env, we use a self-created image based on Alpine...

[xavierfacq]

Hi,

Ok, did you solve your problem ? I don't known Docker very well but in my various, I used to define values by a command argument in the Dockerfile:

-e "transport.host=172.17.0.2"

or

by copying a file

ADD localconfdir/elasticsearch.yml /usr/share/elasticsearch/config/

bye,
Xavier

[morningspace]

Hi @xavierfacq it's fine to use ENV or ADD in Dockerfile, either way is ok... Just different ways to define environment variable. The point that makes me curious is that, after I define the environment variable:

1. Do I have to explicitly reference it in elasticsearch.yml, or...
2. Elasticsearch can recognize it w/o adding that into elasticsearch.yml...

Based on what I tested locally, it seems 2) is always true, until I got the issue a couple of days ago that I described above... I found in another env, not my local env, reindex.remote.whitelist is empty even I set it as environment variable... So, I have to go back to 1). That seems the official way that's documented on Elasticsearch online documentation. Something like:

export reindex.remote.whitelist=blablabla

vs.

export REINDEX_REMOTE_WHITELIST=blablabla
+
reindex.remote.whitelist: "${REINDEX_REMOTE_WHITELIST}" # in elasticsearch.yml

Ideally, I should have figured out the magic behind by going through the Elasticsearch code, but I am still new to the code... it seems not very easy to understand the overall flow of how that's picking up the environment variable for its configuration.