Hi
I have a cluster with 2 nodes of Elasticsearch. TLS/SSL is enables according to the bellow blog:
Everything is OK and monitor-node-01 with IP 192.168.11.142 and monitor-node-02 with IP 192.168.11.143 works correctly with TLS/SSL. But now, I want to add a new node to this cluster.
I used /usr/share/elasticsearch/bin/elasticsearch-certutil to create new certificate for the new node and problem not solved:
elasticsearch-certutil cert --name node3 --dns node3.example.com
Certuril gave me ELK03-SOFT.p12 and I extracted public and private key:
openssl pkcs12 -in ELK03-SOFT.p12 -nocerts -out ELK03-SOFT.key
openssl pkcs12 -in ELK03-SOFT.p12 -clcerts -nokeys -out ELK03-SOFT.crt
In monitor-node-03 the elasticsearch.yml is:
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: /path/to/node3.key
xpack.security.transport.ssl.certificate: /path/to/node3.crt
xpack.security.transport.ssl.certificate_authorities: /path/to/ca.crt
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: /path/to/node3.key
xpack.security.http.ssl.certificate: /path/to/node3.crt
xpack.security.http.ssl.certificate_authorities: /path/to/ca.crt
The problem persists, then I used CA of other 2 nodes that created before(according to the above blog) ca.cert and ca.key that was on monitor-node-01 and monitor-node-02, beside, I created a public and private key with bellow ways and singed them with CA
openssl req -new -newkey rsa:2048 -keyout ELK03.key -out ELK03.csr
openssl x509 -req -in ELK03.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out ELK03.crt -days 365
openssl genpkey -algorithm RSA -out ELK03.key -aes256 -pass pass:
openssl genpkey -algorithm RSA -out ELK03.key -aes256
openssl genpkey -algorithm RSA -out ELK03.key -aes256
openssl req -new -key ELK03.key -out ELK03.csr
openssl x509 -in ./cert.pem -noout -modulus | openssl sha1
openssl rsa -in ./privkey.pem -noout -modulus | openssl sha1
The new Elasticsearch is monitor-node-03 with IP 192.168.11.144. This is the log of new Elasticsearch:
[2023-10-18T17:05:49,479][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [monitor-node-03] client did not trust this server's certificate, closing connection Netty4TcpChannel{localAddress=/192.168.11.144:58800, remoteAddress=ELK01-SOFT/192.168.11.142:9300, profile=default}
On the master node monitor-node-1 when ELK03 (the new node) try to connect shows the bellow log:
[2023-10-21T14:09:56,596][WARN ][o.e.t.TcpTransport ] [monitor-node-01] exception caught on transport layer [Netty4TcpChannel{localAddress=/192.168.11.142:9300, remoteAddress=/192.168.11.144:46822, profile=default}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors
This is my master config:
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
cluster.name: Monitoring_Service
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
node.name: monitor-node-01
#node.master: true
#node.data: true
node.roles: [ master, data, data_hot, data_cold, ingest, transform, ml, remote_cluster_client ]
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
bootstrap.memory_lock: false
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# By default Elasticsearch is only accessible on localhost. Set a different
# address here to expose this node on the network:
#
network.host: 0.0.0.0
transport.host: ELK01-SOFT
# By default Elasticsearch listens for HTTP traffic on the first free port it
# finds starting at 9200. Set a specific HTTP port here:
#
http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
discovery.zen.minimum_master_nodes: 1
discovery.zen.ping.unicast.hosts: ["ELK01-SOFT","ELK02-SOFT","ELK03-SOFT"]
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.seed_hosts: ["host1", "host2"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: ["node-1", "node-2"]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true
#
# ---------------------------------- Security ----------------------------------
#
# *** WARNING ***
#
# Elasticsearch security features are not enabled by default.
# These features are free, but require configuration changes to enable them.
# This means that users don’t have to provide credentials and can get full access
# to the cluster. Network connections are also not encrypted.
#
# To protect your data, we strongly encourage you to enable the Elasticsearch security features.
# Refer to the following documentation for instructions.
#
# https://www.elastic.co/guide/en/elasticsearch/reference/7.16/configuring-stack-security.html
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/node1.key
xpack.security.transport.ssl.certificate: certs/node1.crt
xpack.security.transport.ssl.certificate_authorities: certs/ca.crt
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/node1.key
xpack.security.http.ssl.certificate: certs/node1.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.ml.enabled: true
The ELK02-SOFT is the second node and new node is ELK03-SOFT
How can I join this node to the cluster?