How to add new node to Elasticsearch cluster with Https

Elastic Stack Elasticsearch
1

Hi,

I'm using a single-node Elasticsearch instance.
I want to add another node to my cluster by Enrolling the new node. But I have the same Issue:
ERROR: Failed to determine the health of the cluster with elasticsearch-create-enrollment-token.

How can I add another node to the Cluster? I found no outer method to do that.
Or how can I change my configuration?

Her is the security part of the config from the existing node(cluster).

xpack.security.enabled: true
#xpack.monitoring.collection.enabled: true
xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  key: certs/xxxx.key
  certificate: certs/xxxx.cer
#  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  key: certs/xxxx.key
  certificate: certs/xxxx.cer
2

Have you added the required configuration properties to make your single node cluster to n-node cluster? Please follow: Start the Elastic Stack with security enabled automatically | Elasticsearch Guide [8.6] | Elastic

3

Thanks for your reply!
I've looked up the required configuration. And I changed the transport.host to 0.0.0.0. And I found no more required configurations.
but this doesn't work for me.
I also tried to give a URL to the enrollment command:

/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node --url "https://FQDN:9200"

when I run this I get the flooring Error:

Unable to create enrollment token for scope [node]

ERROR: Unable to create an enrollment token. Elasticsearch node HTTP layer SSL configuration is not configured with a Keystore

It seems I need a Keystore but I thought the Keystore is autogenerated.

But when I try to run the command without a URL he's trying to use an SLL connection with the node IP. but obviously, that doesn't work because the certificate is issued to the domain.

4

You don't have to provide the URL when generating the enrollment token and must be generated from the first/old node. But if you are providing, it should be the URL of old node.
Also, according to elasticsearch-create-enrollment-token | Elasticsearch Guide [8.6] | Elastic, you need to ensure file realm is not disabled in ES and ES_PATH_CONF variable is defined if you are not using default configuration file.

5

I've not moved the default(config in /etc/elasticsearch).

elasticsearch-create-enrollment-token can only be used with Elasticsearch clusters that have been auto-configured for security.

I have found this comment ^ at elasticsearch-create-enrollment-token.

I'm not sure but this means the elasticsearch-create-enrollment-token work only with the default Certs, right?

The problem is I can't use the default Certs due to Security issues.

6

OK, in that case you need to configure your ES (elasticsearch.yml) with SSL properties as described here: Set up basic security for the Elastic Stack plus secured HTTPS traffic | Elasticsearch Guide [8.6] | Elastic
Since you already have the certs available, just follow the steps for new node and update the config yml for both nodes. Also, please make sure you are setting discovery and initial_master_nodes property in new node. Please refer to : Add and remove nodes in your cluster | Elasticsearch Guide [8.6] | Elastic

7

Thanks,
this works sort of.
now I'm getting on the new node a discovery warning and he can't join

[2023-01-16T11:48:49,791][WARN ][o.e.c.c.ClusterFormationFailureHelper] [node-2(new node) HOSTNAME] master not discovered yet, this node has not previously joined a bootstrapped cluster,
and this node must discover master-eligible nodes [node-1(old Note) HOSTNAME] to bootstrap a cluster: have discovered [{node-2(new node) HOSTNAME}{XXXXXXX}{XXXXXX}
{node-2(new node) HOSTNAME}{node-2(new node) IP}{node-2(new node) IP AND PORT}{XXXXX}, {node-1(old Note) HOSTNAME}{XXXXXXX}{XXXXXXX}{node-1(old Note) HOSTNAME}{node-1(old Note) IP}{node-1(old Note) IP AND PORT}
{XXXXXXX}]; discovery will continue using [node-1(old Note) IP AND PORT, node-2(new node) IP AND PORT] from hosts providers and [{node-2(new node) HOSTNAME}{XXXXXX}{XXXXXX}
{node-2(new node) HOSTNAME}{node-2(new node) IP}{node-2(new node) IP AND PORT}{XXXXX}] from last-known cluster state; node term 29, last-accepted version 0 in term 0; joining 
[{node-1(old Note) HOSTNAME}{XXXXXX}{XXXXXX}{node-1(old Note) HOSTNAME}{node-1(old Note) IP}{node-1(old Note) IP AND PORT}{XXXXXXX}]
in term [29] has status [waiting for response] after [9.8s/9806ms]

I found in Bootstrapping a cluster | Elasticsearch Guide [7.17] | Elastic

[master-a.example.com] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [master-a, master-b] to bootstrap a cluster: have discovered [{master-b.example.com}{...
This message shows the node names master-a.example.com and master-b.example.com as well as the cluster.initial_master_nodes entries master-a and master-b , and it is clear from this message that they do not match exactly.

I've changed my config to (I have to do some things in the xpack.security.transport.ssl to make it work to this point, otherwise I got a certificate Error. not optimal but I have found no solution):
Old Note/new note:

discovery.seed_hosts: ["Old note", "new Note"]
cluster.initial_master_nodes: ["Old Note"]
xpack.security.transport.ssl:
  verification_mode: none
  client_authentication: none

And I used the same names in discovery.seed_hosts and cluster.initial_master_nodes on both sides

8

Right, I guess somewhere in 7x versions, Transport SSL became a mandate for n-node clusters, so you need to configure Transport SSL so your nodes can talk to each other.

Also to highlight, ssl.transport must be enabled else your nodes won't be able to join the cluster.

You can use elasticsearch-certutil which is bundled along with elasticsearch and is available in /bin folder. You can either generate P12 certificates and use them or use SAN based ca/instance/key certificates. Documentation is available here: elasticsearch-certutil | Elasticsearch Guide [8.6] | Elastic

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.