The values for placeholders are taken from env variables but for example for DEV environment I'd like to have single-node discovery type and for QA we want to have cluster.
Currently when I try to pass discovery-type as single and other properties (related to clustering) as empty string I get an error: "Could not resolve placeholder ' NAME OF PLACEHOLDER' " when I try to put something into that plaeholder then I get: "[cluster.initial_master_nodes] is not allowed when [discovery.type] is set to [single-node]"
Is it possible to achieve what we're aiming for using single configuration.yml file with placeholders or is it neccessary to create configuration file per environment (or per clustered/non-clustered config) ?
Shouldnt you get away with not changing the discovery.type to "single-node" for your single-node case?
Using the cluster discovery.type with the discovery.seed_hosts and cluster.initial_master_nodes only containing your single node should suffice to bootstrap the "cluster".
Without having tested it the doc´s make it seem so:
It is technically sufficient to set cluster.initial_master_nodes on a single master-eligible node in the cluster, and only to mention that single node in the setting’s value, but this provides no fault tolerance before the cluster has fully formed
The cluster will boostrap and start it seems, obviously the multi node resilience is missing until "the cluster is fully formed" but thats to be expected with a single node. But to be able to not provide fault tolerance the "cluster" has to be running in the first place, before having formed/connected with other nodes Which makes me think it works.
discovery.type
Specifies whether Elasticsearch should form a multiple-node cluster. By default, Elasticsearch discovers other nodes when forming a cluster and allows other nodes to join the cluster later. If discovery.type is set to single-node , Elasticsearch forms a single-node cluster and suppresses the timeouts set by cluster.publish.timeout and cluster.join.timeout . For more information about when you might use this setting, see Single-node discovery.
The discovery.type doc makes it seem like you will receive timeouts in this single node dispite cluster formation case but that should be just about it.
After the "cluster.publish.timeout" happened on the single node, the cluster should start is my guess.
Depending on your deployment process you could also overwrite the settings of the elasticsearch.yml (and all other config files in the ELK realm) on startup by adding command line parameters to their execution.
My node.name is derived from docker (The discovery seed host as well, in my case the discovery seed host contains the ips of all running docker containers, in this single node case this corrosponds to the "discovery.seed_hosts" setting beeing set to a single machine, the one the single node runs on where it can discover it self if it wants to. Which it does not want to btw duo to all nodes needed for the election process defined in the "initial master nodes" being present.), my single nodes name is "docker-desktop" which also is the only "cluster.initial_master_node/s".
As you can see I did not touch the "discovery.type" setting at all. The single node will bootstrap a cluster by electing it self as seen in the CLI screenshot above.
If you want a setup with multiple nodes discovering each other you only have to change the "cluster.initial_master_nodes" through an environmental or similier. That way you extend a list of one to multiple and dont have to deal with removing setting nodes them self.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
Which makes me think it works.