I'm trying to get a second elasticsearch cluster running on a machine which already has a cluster. I have 7.9.2 installed on Ubuntu 18.04, and it's working properly. I created a new service file for the new cluster, new config folder, log and data folders, but everytime I try starting the new cluster it always seems to want to start a second instance of the first cluster. ie: it's trying to obtain a lock on the data path for the first cluster.
Through much trial and error I've finally come to the realization that /usr/share/elasticsearch/bin/elasticsearch-env is hard-coded to source /etc/default/elasticsearch.
So the only way I can start a second cluster so far is to start the first one, modify /etc/default/elasticsearch with the config path for the new cluster, and then start the new cluster.
At that point I have two clusters running on one machine, independent of each other. But now the problem I have is if I stop/start the old cluster. It now fails because of the modified /etc/default/elasticsearch.
Can anyone provide advice on how to get 2 clusters running on one box?