Not able to Add Node to Existing Cluster

Debasis_Mallick · November 27, 2023, 4:29pm

Hi Team,

I had requirement to create two node cluster of 8.9.2 version. we did Elasticsearch installation on both nodes through RPM based.
I had generated enrollment token (elasticsearch-create-enrollment-token -s node
) from node1 but while adding the node 2 with "elasticsearch-reconfigure-node " command by passing the enrollment-token which is generated from node-1 getting below error.

/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token eyJ2ZXIiOiI4LjkuMiIsImFkciI6WyIxMC4xMC4xOC4xNzQ6OTIwMCJdLCJmZ3IiOiJkMDA4MzFjZDhhZDZlODVkNGRiNDUzYThmM2ExNTk4YWVmNjQ0ZWRlYmFmZGQxZWI0MmI5OTMxODljZThlYTUzIiwia2V5IjoiaHhoMUVZd0JLMGg2a0VEZkpoc0Y6
ERROR: Aborting enrolling to cluster. This node doesn't appear to be auto-configured for security. Expected configuration is missing from elasticsearch.yml.

Could you please help me how I can add node-2 so that these two nodes act as a cluster.

Thanks,
Debasis

stephenb · November 27, 2023, 6:18pm

Hi @Debasis_Mallick

Did you follow the instructions here, including making sure the transport protocol is bound to a non-local host.

Also, can you share the elasticsearch.yml for node 2

Debasis_Mallick · November 28, 2023, 7:34am

Hi @stephenb thanks for response.

1.I had installed Elasticsearch 8.9.2 on both nodes (cb2 and cb3)through RPM as non-root user(elasticsearch).
sudo rpm -ivh elasticsearch-8.9.2-x86_64.rpm

Then update the elasticsearch.yml file with proper values in cb2 node and start elasticsearch service through systemctl.

3.Generate enrollment token from cb-2.

[root@cb-2 bin]# /usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node
eyJ2ZXIiOiI4LjkuMiIsImFkciI6WyIxMC4xMC4xOC4xNzQ6OTIwMCJdLCJmZ3IiOiJhOGUxYTY1OTRmOWVkZmM2NGIxNDc4Y2IyNmUxZWU2MjhhY2YzNzYxNzQ0NDZhNmRmMjFjODg5NTRjN2RiMmU0Iiwia2V5IjoiaUxQTUZJd0JUMjBfaENWRTFGX2M6ZWswWWEycW5UYy1hTGNrZkY3SFFvdyJ9

4.Then login to cb-3 and trying to enroll the node by token generated in step-3.

[root@cb-3 elasticsearch]# /usr/share/elasticsearch/bin/elasticsearch --enrollment-token eyJ2ZXIiOiI4LjkuMiIsImFkciI6WyIxMC4xMC4xOC4xNzQ6OTIwMCJdLCJmZ3IiOiJhOGUxYTY1OTRmOWVkZmM2NGIxNDc4Y2IyNmUxZWU2MjhhY2YzNzYxNzQ0NDZhNmRmMjFjODg5NTRjN2RiMmU0Iiwia2V5IjoiaUxQTUZJd0JUMjBfaENWRTFGX2M6ZWswWWEycW5UYy1hTGNrZkY3SFFvdyJ9
ERROR: Skipping security auto configuration because it appears that security is already configured.

I had attached cb-3 yml file . Could you please check and advise.

[root@cb-3 elasticsearch]# cat elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
#network.host: 192.168.0.1
#
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Allow wildcard deletion of indices:
#
#action.destructive_requires_name: false

#----------------------- BEGIN SECURITY AUTO CONFIGURATION -----------------------
#
# The following settings, TLS certificates, and keys have been automatically
# generated to configure Elasticsearch security features on 28-11-2023 07:05:28
#
# --------------------------------------------------------------------------------

# Enable security features
xpack.security.enabled: true

xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
  enabled: true
  keystore.path: certs/http.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
# Create a new cluster with the current node only
# Additional nodes can still join the cluster later
cluster.initial_master_nodes: ["cb-3"]

# Allow HTTP API connections from anywhere
# Connections are encrypted and require user authentication
http.host: 0.0.0.0

# Allow other nodes to join the cluster from anywhere
# Connections are encrypted and mutually authenticated
#transport.host: 0.0.0.0

#----------------------- END SECURITY AUTO CONFIGURATION -------------------------
[root@cb-3 elasticsearch]#

Christian_Dahlqvist · November 28, 2023, 7:50am

You need to uncomment the transport host setting. By default it uses localhost and the nodes use this to communicate.

Debasis_Mallick · November 28, 2023, 9:30am

@Christian_Dahlqvist I had uncommented the "transport.host" but still same error while adding the node.

[root@cb-3 bin]# ./elasticsearch --enrollment-token eyJ2ZXIiOiI4LjkuMiIsImFkciI6WyIxMC4xMC4xOC4xNzQ6OTIwMCJdLCJmZ3IiOiJhOGUxYTY1OTRmOWVkZmM2NGIxNDc4Y2IyNmUxZWU2MjhhY2YzNzYxNzQ0NDZhNmRmMjFjODg5NTRjN2RiMmU0Iiwia2V5IjoiaXJOQ0ZZd0JUMjBfaENWRVBWX3g6VGpnZDAwdXRRVUNRSWNTMTBiS0JWQSJ9

ERROR: Skipping security auto configuration because it appears that security is already configured.

Thanks,
Debasis

Debasis_Mallick · November 28, 2023, 9:56am

@stephenb As per documentation it is mentioned that Elasticsearch automatically generates certificate in certs folder of node2 (cb-3).

config\certs

But my case the folder already created after the installation of Elasticsearch in node2 (cb-3) so is there anything wrong in installation process.

Thanks,
Debasis

stephenb · November 28, 2023, 4:29pm

Apologies I am a bit confused... You s

Do you have transport.host set to 0.0.0.0

And

To add a node to a cluster running on multiple machines, you must also set discovery.seed_hosts so that the new node can discover the rest of its cluster.

So do you have discovery.seed_hosts set?

Can you share your latest.yml

Debasis_Mallick · November 28, 2023, 4:35pm

Thanks @stephenb after adding discovery.seed_hosts in elasticsearch.yml and then able to add node as below

[root@cb-3 ~]# /usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token eyJ2ZXIiOiI4LjkuMiIsImFkciI6WyIxMC4xMC4xOC4xNzQ6OTIwMCJdLCJmZ3IiOiJhOGUxYTY1OTRmOWVkZmM2NGIxNDc4Y2IyNmUxZWU2MjhhY2YzNzYxNzQ0NDZhNmRmMjFjODg5NTRjN2RiMmU0Iiwia2V5IjoiTDBKUkZvd0JPT2RiY0tGRG1CZlU6eFlPd2hpbWpSRk9CUWsxcVRuMGNoUSJ9

This node will be reconfigured to join an existing cluster, using the enrollment token that you provided.
This operation will overwrite the existing configuration. Specifically:
  - Security auto configuration will be removed from elasticsearch.yml
  - The [certs] config directory will be removed
  - Security auto configuration related secure settings will be removed from the elasticsearch.keystore
Do you want to continue with the reconfiguration process [y/N]y

Thanks,
Debasis

system · December 26, 2023, 4:36pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.