"node.name" value not being applied

[mcohen (Mark Cohen)]

I'm testing with an internal cluster, 3 nodes on the same VM.
Env: Ubuntu 18.04.3 LTS, elasticsearch-7.3.2-amd64.deb (Debian)

To simulate multiple nodes, I've replicated the /etc/elasticsearch directory, results are as follows:
/etc/elasticsearch
/etc/elasticsearch-node-2
/etc/elasticsearch-node-3

Here's the config for one of the elasticsearch.yml files (ie: /etc/elasticsearch/elasticsearch.yml):

cluster.name: new-tokyo
node.name: akira
path.data: /opt/elasticsearch-data
path.logs: /opt/elasticsearch-logs
network.host: 0.0.0.0
http.port: 9200
cluster.initial_master_nodes: ["akira", "tetsuo", "kaneda"]
node.max_local_storage_nodes: 3

The only difference between the 3 elasticsearch.yml files is:

• node.name value (akira, tetsuo, kaneda)
• http.port value (9200, 9201, 9202)

jvm.options is identical for all three nodes. (512m)

If I start the nodes and pass the value from the command line (ie: ES_PATH_CONF=/etc/elasticsearch-node-2/) then the node names are unique.

If I setup and start elasticsearch as a service (ie: systemctl start elasticsearch.service), the node names are all the same, akira.

Whats particularly confusing is that I can change other values, such as http.port and the change is reflected (once restarted). Is this data cached somewhere? I've tried trashing the data folder (stock location /var/lib/elasticsearch/nodes and my custom location /opt/elasticsearch-data/nodes/), to no avail, and have literally spent days trying to figure this out, and have tried with 7.3, and 7.4. Any assistance is appreciated, thanks.

[mcohen (Mark Cohen)]

Here's an example of the relevant errors I'm seeing in the logs, "MasterNotDiscoveredException", etc...

[akira] bootstrapping cancelled java.lang.IllegalStateException: requirement [akira] matches multiple nodes:

and

this node must discover master-eligible nodes [akira, tetsuo, kaneda] to bootstrap a cluster: have discovered [{akira}