# Setting up 3 node cluster for first time and getting errors

**URL:** <https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775>\
**Category:** Elasticsearch\
**Created:** [June 1, 2025, 3:17pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775 "2025-06-01T15:17:54Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 1, 2025, 3:17pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/1 "2025-06-01T15:17:54Z")

</div>

I am trying to setup a 3 node cluster on RHEL 8.10 VMs with elasticsearch-8.18.1-linux-x86\_64.tar.gz. The ES\_JAVA\_HOME is configured with jdk-21.0.5

I have followed the steps from [Set up HTTPS | Elastic Docs](https://www.elastic.co/docs/deploy-manage/security/set-up-basic-security-plus-https#encrypt-http-communication) and generated http-ca.p12

My primary master elasticsearch.yml looks like the below:

```auto
cluster.name: DevAdvSearch
node.name: DevES1
discovery.seed_hosts: ["xx.xxx.xxx.xx","xx.xxx.xxx.xx","xx.xxx.xxx.xx"]
cluster.initial_master_nodes: ["DevES1"]
node.roles: [master]
path.data: /appln_storage/data
path.logs: /appln_storage/logs
http.port: 9200
network.host: 0.0.0.0
transport.host: 0.0.0.0
xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  keystore.type: PKCS12
  keystore.path: http-ca.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.type: PKCS12
  keystore.path: http-ca.p12

```

The other 2 data node's elasticsearch.yml looks like below:

```auto
cluster.name: DevAdvSearch
node.name: DevES2 (DevES3 for the 3rd node)
discovery.seed_hosts: ["xx.xxx.xxx.xx","xx.xxx.xxx.xx","xx.xxx.xxx.xx"]
cluster.initial_master_nodes: ["DevES1"]
node.roles: [data]
path.data: /appln_storage/data
path.logs: /appln_storage/logs
http.port: 9200
network.host: 0.0.0.0
transport.host: 0.0.0.0
xpack.security.enabled: true
xpack.security.enrollment.enabled: true

xpack.security.http.ssl:
  enabled: true
  keystore.type: PKCS12
  keystore.path: http-ca.p12

# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  keystore.type: PKCS12
  keystore.path: http-ca.p12

```

The password for the below were stored at keystore

```auto
xpack.security.http.ssl.keystore.secure_password
xpack.security.transport.ssl.keystore.secure_password
xpack.security.transport.ssl.truststore.secure_password

```

**Problem Statement:**  
When I run ./bin/elasticsearch from the master node, was getting couple of warnings and the below from log:

> [2025-06-01T14:22:19,662][WARN][o.e.c.c.Coordinator] [DevES1] This node is a fully-formed single-node cluster with cluster UUID [jHdfrWBySPqGSmlv58-0Ew], but it is configured as if to discover other nodes and form a multi-node cluster via the [discovery.seed\_hosts=[xx.xxx.xxx.xx,xx.xxx.xxx.xx,xx.xxx.xxx.xx]] setting. Fully-formed clusters do not attempt to discover other nodes, and nodes with different cluster UUIDs cannot belong to the same cluster. The cluster UUID persists across restarts and can only be changed by deleting the contents of the node's data path(s). Remove the discovery configuration to suppress this message. See [[Bootstrapping a cluster | Elasticsearch Guide [8.18] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/modules-discovery-bootstrap-cluster.html#modules-discovery-bootstrap-cluster-joining)] for more information.  
> [2025-06-01T14:22:24,876][ERROR][o.e.x.d.l.DeprecationIndexingComponent] [DevES1] Bulk write of deprecation logs encountered some failures: [[Z\_TeK5cBg1URBLnUZm8E org.elasticsearch.action.UnavailableShardsException: [.ds-.logs-deprecation.elasticsearch-default-2025.06.01-000001][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.ds-.logs-deprecation.elasticsearch-default-2025.06.01-000001][0]] containing [index {[.logs-deprecation.elasticsearch-default][Z\_TeK5cBg1URBLnUZm8E], source[{"@timestamp":"2025-06-01T14:20:49.772Z", "log.level": "WARN", "data\_stream.dataset":"deprecation.elasticsearch","data\_stream.namespace":"default","data\_stream.type":"logs","elasticsearch.event.category":"indices","event.code":"dot-prefix","message":"Index [.monitoring-es-8-\*] name begins with a dot (.), which is deprecated, and will not be allowed in a future Elasticsearch version." , "ecs.version": "1.2.0","service.name":"ES\_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[DevES1][generic][T#8]","log.logger":"org.elasticsearch.deprecation.validation.DotPrefixValidator","elasticsearch.cluster.uuid":"jHdfrWBySPqGSmlv58-0Ew","elasticsearch.node.id":"7NkAJJsCT12KZCGud0CXFg","elasticsearch.node.name":"DevES1","elasticsearch.cluster.name":"DevAdvSearch"}  
> ]}]]]]  
> [2025-06-01T14:22:28,948][ERROR][o.e.x.s.InitialNodeSecurityAutoConfiguration] [DevES1] Failed to generate credentials for the elastic built-in superuserorg.elasticsearch.action.UnavailableShardsException: [.security-7][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.security-7][0]] containing [index {[.security][reserved-user-elastic], source[{"password":"xxxxxxxxxxxxxxxxxxx","enabled":true,"type":"reserved-user"}]}] and a refresh]

See logs for more details.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━  
✅ Elasticsearch security features have been automatically configured!  
✅ Authentication is enabled and cluster connections are encrypted.

❌ Unable to auto-generate the password for the elastic built-in superuser.

ℹ HTTP CA certificate SHA-256 fingerprint:

* * *

❌ Unable to generate an enrollment token for Kibana instances, try invoking `bin/elasticsearch-create-enrollment-token -s kibana`.

❌ An enrollment token to enroll new nodes wasn't generated. To add nodes and enroll them into this cluster:  
• On this node:  
⁃ Create an enrollment token with `bin/elasticsearch-create-enrollment-token -s node`.  
⁃ Restart Elasticsearch.  
• On other nodes:  
⁃ Start Elasticsearch with `bin/elasticsearch --enrollment-token <token>`, using the enrollment token that you generated.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

> [2025-06-01T14:22:29,936][ERROR][o.e.x.i.h.ILMHistoryStore] [DevES1] failures: [{afTeK5cBg1URBLnUeW\_L=org.elasticsearch.action.UnavailableShardsException: [.ds-ilm-history-7-2025.06.01-000001][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.ds-ilm-history-7-2025.06.01-000001][0]] containing [3] requests], avTeK5cBg1URBLnUeW\_L=org.elasticsearch.action.UnavailableShardsException: [.ds-ilm-history-7-2025.06.01-000001][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.ds-ilm-history-7-2025.06.01-000001][0]] containing [3] requests], aPTeK5cBg1URBLnUeW\_K=org.elasticsearch.action.UnavailableShardsException: [.ds-ilm-history-7-2025.06.01-000001][0] primary shard is not active Timeout: [1m], request: [BulkShardRequest [[.ds-ilm-history-7-2025.06.01-000001][0]] containing [3] requests]}]  
> [2025-06-01T14:22:34,429][WARN][o.e.h.AbstractHttpServerTransport] [DevES1] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/xx.xxx.xxx.xx:9200, remoteAddress=/xx.xx.xxx.xx:55135}io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: illegal\_parameter  
> at io.netty.codec@4.1.118.Final/io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:500)  
> at io.netty.codec@4.1.118.Final/io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)  
> See logs for more details.  
> [2025-06-01T14:22:34,484][WARN][o.e.h.AbstractHttpServerTransport] [DevES1] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/xx.xxx.xxx.xx:9200, remoteAddress=/xx.xx.xxx.xx:55136}io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: illegal\_parameter  
> at io.netty.codec@4.1.118.Final/io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:500)  
> at io.netty.codec@4.1.118.Final/io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)  
> See logs for more details.  
> [2025-06-01T14:40:19,710][WARN][o.e.c.c.Coordinator] [DevES1] This node is a fully-formed single-node cluster with cluster UUID [jHdfrWBySPqGSmlv58-0Ew], but it is configured as if to discover other nodes and form a multi-node cluster via the [discovery.seed\_hosts=[xx.xxx.xxx.xx,xx.xxx.xxx.xx,xx.xxx.xxx.xx]] setting. Fully-formed clusters do not attempt to discover other nodes, and nodes with different cluster UUIDs cannot belong to the same cluster. The cluster UUID persists across restarts and can only be changed by deleting the contents of the node's data path(s). Remove the discovery configuration to suppress this message. See [[Bootstrapping a cluster | Elasticsearch Guide [8.18] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/modules-discovery-bootstrap-cluster.html#modules-discovery-bootstrap-cluster-joining)] for more information.

When I tried generating the enrollment token from the other window of same VM which I am running, getting the below error  
`$ ./bin/elasticsearch-create-enrollment-token -s node`

> warning: ignoring JAVA\_HOME=/usr/java/jdk17; using ES\_JAVA\_HOME  
> 14:00:51.025 [main] WARN org.elasticsearch.common.ssl.DiagnosticTrustManager - failed to establish trust with server at [xx.xxx.xxx.xx]; the server provided a certificate with subject name [CN=Elastic Certificate Tool Autogenerated CA], fingerprint [xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx], keyUsage [keyCertSign, cRLSign] and no extendedKeyUsage; the certificate is valid between [2025-06-01T12:37:58Z] and [2028-05-31T12:37:58Z] (current time is [2025-06-01T14:00:51.022816617Z], certificate dates are valid); the session uses cipher suite [TLS\_AES\_256\_GCM\_SHA384] and protocol [TLSv1.3]; the certificate does not have any subject alternative names; the certificate is self-issued; the [CN=Elastic Certificate Tool Autogenerated CA] certificate is trusted in this ssl context ([xpack.security.http.ssl (with trust configuration: Composite-Trust{JDK-trusted-certs,StoreTrustConfig{path=http-ca.p12, password=, type=PKCS12, algorithm=PKIX}})])  
> java.security.cert.CertificateException: No subject alternative names present  
> at sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:138) ~[?:?]  
> at sun.security.util.HostnameChecker.match(HostnameChecker.java:101) ~[?:?]  
> at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:466) ~[?:?]  
> at sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:432) ~[?:?]  
> at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:237) ~[?:?]  
> at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:?]  
> at org.elasticsearch.common.ssl.DiagnosticTrustManager.checkServerTrusted(DiagnosticTrustManager.java:81) ~[?:?]  
> at sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1302) ~[?:?]  
> at sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1195) ~[?:?]  
> at sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1138) ~[?:?]  
> at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:393) ~[?:?]  
> at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:476) ~[?:?]  
> at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:447) ~[?:?]  
> at sun.security.ssl.TransportContext.dispatch(TransportContext.java:206) ~[?:?]  
> at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?]  
> at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1506) ~[?:?]  
> at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1421) ~[?:?]  
> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:455) ~[?:?]  
> at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:426) ~[?:?]  
> at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:586) ~[?:?]  
> at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:187) ~[?:?]  
> at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:141) ~[?:?]  
> at org.elasticsearch.xpack.core.common.socket.SocketAccess.lambda$doPrivileged$0(SocketAccess.java:42) ~[?:?]  
> at java.security.AccessController.doPrivileged(AccessController.java:571) ~[?:?]  
> at org.elasticsearch.xpack.core.common.socket.SocketAccess.doPrivileged(SocketAccess.java:41) ~[?:?]  
> at org.elasticsearch.xpack.core.security.CommandLineHttpClient.execute(CommandLineHttpClient.java:178) ~[?:?]  
> at org.elasticsearch.xpack.core.security.CommandLineHttpClient.execute(CommandLineHttpClient.java:112) ~[?:?]  
> at org.elasticsearch.xpack.security.tool.BaseRunAsSuperuserCommand.checkClusterHealthWithRetries(BaseRunAsSuperuserCommand.java:214) ~[?:?]  
> at org.elasticsearch.xpack.security.tool.BaseRunAsSuperuserCommand.execute(BaseRunAsSuperuserCommand.java:127) ~[?:?]  
> at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:55) ~[elasticsearch-8.18.1.jar:8.18.1]  
> at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:101) ~[elasticsearch-cli-8.18.1.jar:8.18.1]  
> at org.elasticsearch.cli.Command.main(Command.java:54) ~[elasticsearch-cli-8.18.1.jar:8.18.1]  
> at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:65) ~[cli-launcher-8.18.1.jar:8.18.1]  
> ERROR: Failed to determine the health of the cluster. , with exit code 69

Would like to understand what is missing and how to get this fixed.

Appreciate your help!!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 1, 2025, 3:36pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/2 "2025-06-01T15:36:25Z")

</div>

Hello and welcome,

You have only one node with the `master` role, the `discovery.seed_hosts` is a list of master eligible nodes, so it needs to haver just the master eligible node, remove the non-eligible nodes from this list.

Did you manually generated the certificates? If true you need to remove this `xpack.security.enrollment.enabled: true` from the configuration file, the enrollment process only works if security is auto-configured.

Also, you have 3 nodes, why use just one as a master? Your cluster will not be resilient to failures.

---

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 2, 2025, 3:24pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/3 "2025-06-02T15:24:38Z")

</div>

Thank you for your inputs. Much appreciated!!

I tried by setting initial cluster with all 3 nodes and had all 3 nodes in initial\_master\_nodes as below:

```auto
cluster.name: DevAdvSearch
node.name: DevES1
path.data: /appln_storage/data
path.logs: /appln_storage/logs
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["xx.xxx.xxx.xx","xx.xxx.xxx.xx","xx.xxx.xxx.xx"]
cluster.initial_master_nodes: ["DevES1", "DevES2", "DevES3"]
xpack.security.enabled: false
xpack.security.enrollment.enabled: true
xpack.security.transport.ssl.enabled: false

```

However, I was getting the below warnings and the master was not discovered for long time and I had to stop.

> [2025-06-02T15:17:35,092][WARN][o.e.c.c.ClusterFormationFailureHelper] [DevES1] master not discovered yet, this node has not previously joined a bootstrapped cluster, and this node must discover master-eligible nodes [DevES1, DevES2, DevES3] to bootstrap a cluster: have discovered [{DevES1}{T4OIv2HURpa-sXKBlaAyxw}{nrhvggrqRNuGa74NmRchFQ}{DevES1}{xx.xxx.xxx.xx}{xx.xxx.xxx.xx:9300}{cdfhilmrstw}{8.18.1}{7000099-8525000}]; discovery will continue using [xx.xxx.xxx.xx:9300] from hosts providers and [{DevES1}{T4OIv2HURpa-sXKBlaAyxw}{nrhvggrqRNuGa74NmRchFQ}{DevES1}{xx.xxx.xxx.xx}{xx.xxx.xxx.xx:9300}{cdfhilmrstw}{8.18.1}{7000099-8525000}] from last-known cluster state; node term 0, last-accepted version 0 in term 0; for troubleshooting guidance, see [Troubleshooting discovery | Elasticsearch Guide [8.18] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/discovery-troubleshooting.html)  
> [2025-06-02T15:17:35,253][WARN][o.e.d.PeerFinder] [DevES1] address [xx.xxx.xxx.xx:9300], node [unknown] discovery result: [xx.xxx.xxx.xx:9300] connect\_exception: Failed execution: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: /xx.xxx.xxx.xx:9300: Connection refused: /xx.xxx.xxx.xx:9300: Connection refused; for summary, see logs from org.elasticsearch.cluster.coordination.ClusterFormationFailureHelper; for troubleshooting guidance, see [Troubleshooting discovery | Elasticsearch Guide [8.18] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.18/discovery-troubleshooting.html)

Not sure how to proceed further and tried altering the setup with the one I mentioned above. I even tried to proceed with no security - to check if I am able to get the cluster running. But, it didn't work.

Appreciate your help!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 2, 2025, 3:27pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/4 "2025-06-02T15:27:41Z")

</div>

> [@IamGuna](#):
>
> Connection refused: /xx.xxx.xxx.xx:9300: Connection refused: /xx.xxx.xxx.xx:9300

Are all machines able to communicate with each other? This seems like a network issue.

You need to start the Elasticsearch service in the 3 nodes around the same time for the cluster to be formed.

---

<div class="post-metadata">

**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\
**Post date:** [June 2, 2025, 4:00pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/5 "2025-06-02T16:00:55Z")

</div>

> [@leandrojmp](#):
>
> Are all machines able to communicate with each other? This seems like a network issue.

Initially wrote you used RHEL 8 VMs. Make sure the VMs inter-node traffic is allowed.

And check the RHEL firewalld rules/config/status.

---

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 3, 2025, 12:34pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/6 "2025-06-03T12:34:30Z")

</div>

ah... very valid one. Let me check on this.

---

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 4, 2025, 12:05pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/7 "2025-06-04T12:05:14Z")

</div>

Thank you @leandrojmp . I followed your instructions and I was able to run all 3 nodes together. When I tried to enable security, facing the issues.

> [2025-06-04T11:49:34,078][WARN][o.e.x.c.s.t.n.SecurityNetty4Transport] [DevES2] client did not trust this server's certificate, closing connection Netty4TcpChannel{localAddress=/xx.xxx.xxx.xx:9300, remoteAddress=/xx.xxx.xxx.xx:57726, profile=default}  
> [2025-06-04T11:49:34,195][WARN][o.e.c.s.DiagnosticTrustManager] [DevES2] failed to establish trust with server at [xx.xxx.xxx.xx]; the server provided a certificate with subject name [CN=instance], fingerprint [293ba9498cb20fbb2d12d637bda111010d6018d6], no keyUsage and no extendedKeyUsage; the certificate is valid between [2025-06-04T10:24:26Z] and [2028-06-03T10:24:26Z] (current time is [2025-06-04T11:49:34.195517607Z], certificate dates are valid); the session uses cipher suite [TLS\_AES\_256\_GCM\_SHA384] and protocol [TLSv1.3]; the certificate does not have any subject alternative names; the certificate is issued by [CN=Elastic Certificate Tool Autogenerated CA]; the certificate is signed by (subject [CN=Elastic Certificate Tool Autogenerated CA] fingerprint [30a19723d2a8a68e619b36698342d020ad9c14d0] {trusted issuer}) which is self-issued; the [CN=Elastic Certificate Tool Autogenerated CA] certificate is trusted in this ssl context ([xpack.security.transport.ssl (with trust configuration: StoreTrustConfig{path=elastic-certificates.p12, password=, type=PKCS12, algorithm=PKIX})])java.security.cert.CertificateException: No subject alternative names present  
> at java.base/sun.security.util.HostnameChecker.matchIP(HostnameChecker.java:138)  
> at java.base/sun.security.util.HostnameChecker.match(HostnameChecker.java:101)  
> at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:466)  
> at java.base/sun.security.ssl.X509TrustManagerImpl.checkIdentity(X509TrustManagerImpl.java:432)  
> at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:291)  
> See logs for more details.  
> [2025-06-04T11:49:34,197][WARN][o.e.t.TcpTransport] [DevES2] exception caught on transport layer [Netty4TcpChannel{localAddress=/xx.xxx.xxx.xx:57446, remoteAddress=/xx.xxx.xxx.xx:9300, profile=default}], closing connectionio.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: No subject alternative names present  
> at io.netty.codec@4.1.118.Final/io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:500)  
> at io.netty.codec@4.1.118.Final/io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)  
> at io.netty.transport@4.1.118.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)  
> See logs for more details.

My elasticsearch.yml looks like below as I updated it for ssl and transport.

```auto
cluster.name: DevAdvSearch
node.name: DevES1
path.data: /appln_storage/data
path.logs: /appln_storage/logs
network.host: 0.0.0.0
http.port: 9200
transport.port: 9300
discovery.seed_hosts: ["xx.xxx.xxx.xx","xx.xxx.xxx.xx","xx.xxx.xxx.xx"]
cluster.initial_master_nodes: ["DevES1", "DevES2", "DevES3"]

xpack.security.enabled: true
xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: elastic-certificates.p12
xpack.security.http.ssl.keystore.password: plain_text_pass_given

# Enable encryption for transport layer
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.keystore.password: plain_text_pass_given

# Create a new cluster with security artifacts
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.password: plain_text_pass_given

# Enable audit logging
xpack.security.audit.enabled: true

# Allow wildcard deletion
action.destructive_requires_name: false

# ---------------------------------- Monitoring ------------------------------
# Enable monitoring
xpack.monitoring.collection.enabled: true

```

Generated ca, keystore & truststore using the below commands:

```auto
elasticsearch-certutil ca
elasticsearch-certutil cert --ca elastic-stack-ca.p12

```

Could you please guide on how should I proceed?

Appreciate your help!!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 4, 2025, 12:15pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/8 "2025-06-04T12:15:02Z")

</div>

You are missing `xpack.security.transport.ssl.certificate_authorities` and `xpack.security.http.ssl.certificate_authorities` pointing to the CA file that you used to create the certificated.

Something like:

```auto
xpack.security.transport.ssl.certificate_authorities: /path/to/elastic-stack-ca.p12
xpack.security.http.ssl.certificate_authorities: /path/to/elastic-stack-ca.p12

```

---

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 4, 2025, 1:32pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/9 "2025-06-04T13:32:35Z")

</div>

Thank you @leandrojmp . I tried your recommendations and got the below ones.

> [2025-06-04T13:21:27,267][ERROR][o.e.b.Elasticsearch] [DevES2] fatal exception while booting Elasticsearchorg.elasticsearch.ElasticsearchSecurityException: failed to load SSL configuration [xpack.security.http.ssl] - cannot load PEM certificate\_authorities from [/app/elasticsearch-8.18.1/config/elastic-certificates.p12] due to CertificateParsingException (signed fields invalid)  
> at org.elasticsearch.xcore@8.18.1/org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSslConfigurations$11(SSLService.java:622)  
> at java.base/java.util.HashMap.forEach(HashMap.java:1429)  
> at java.base/java.util.Collections$UnmodifiableMap.forEach(Collections.java:1707)  
> at org.elasticsearch.xcore@8.18.1/org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:618)  
> at org.elasticsearch.xcore@8.18.1/org.elasticsearch.xpack.core.ssl.SSLService.(SSLService.java:160)  
> See logs for more details.  
> ERROR: Elasticsearch did not exit normally - check the logs at /appln\_storage/logs/DevES.log  
> ERROR: Elasticsearch died while starting up, with exit code 1

When I checked the log directory, noticed the below warning and error.

> [2025-06-04T13:21:27,245][WARN][o.e.e.r.p.P.(.o.e.server] [DevES2] Not entitled: component [(server)], module [org.elasticsearch.server], class [class org.elasticsearch.watcher.FileWatcher], entitlement [file], operation [read], path [/app/elasticsearch-8.18.1/config/users]  
> org.elasticsearch.entitlement.runtime.api.NotEntitledException: component [(server)], module [org.elasticsearch.server], class [class org.elasticsearch.watcher.FileWatcher], entitlement [file], operation [read], path [/app/elasticsearch-8.18.1/config/users]  
> at org.elasticsearch.entitlement.runtime.policy.PolicyManager.notEntitled(PolicyManager.java:690) ~[elasticsearch-entitlement-8.18.1.jar:?]  
> at org.elasticsearch.entitlement.runtime.policy.PolicyManager.checkFileRead(PolicyManager.java:511) ~[elasticsearch-entitlement-8.18.1.jar:?]  
> at org.elasticsearch.entitlement.runtime.policy.PolicyManager.checkFileRead(PolicyManager.java:475) ~[elasticsearch-entitlement-8.18.1.jar:?]  
> at org.elasticsearch.entitlement.runtime.api.ElasticsearchEntitlementChecker.check$java\_nio\_file\_Files$$exists(ElasticsearchEntitlementChecker.java:2114) ~[elasticsearch-entitlement-8.18.1.jar:?]  
> at java.nio.file.Files.exists(Files.java) ~[?:?]  
> at org.elasticsearch.watcher.FileWatcher.fileExists(FileWatcher.java:120) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.FileWatcher$FileObserver.init(FileWatcher.java:236) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.FileWatcher$FileObserver.createChild(FileWatcher.java:256) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.FileWatcher$FileObserver.listChildren(FileWatcher.java:276) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.FileWatcher$FileObserver.onDirectoryCreated(FileWatcher.java:387) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.FileWatcher$FileObserver.init(FileWatcher.java:241) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.FileWatcher.doInit(FileWatcher.java:74) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.AbstractResourceWatcher.init(AbstractResourceWatcher.java:26) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.watcher.ResourceWatcherService.add(ResourceWatcherService.java:128) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.xpack.core.ssl.SSLConfigurationReloader.lambda$startWatching$2(SSLConfigurationReloader.java:110) ~[?:?]  
> at java.util.HashMap.forEach(HashMap.java:1429) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLConfigurationReloader.startWatching(SSLConfigurationReloader.java:105) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLConfigurationReloader.(SSLConfigurationReloader.java:51) ~[?:?]  
> at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:495) ~[?:?]  
> at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:325) ~[?:?]  
> at org.elasticsearch.node.NodeConstruction.lambda$construct$17(NodeConstruction.java:924) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.plugins.PluginsService.lambda$flatMap$0(PluginsService.java:186) ~[elasticsearch-8.18.1.jar:?]  
> at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]  
> at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]  
> at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722) ~[?:?]  
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]  
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]  
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575) ~[?:?]  
> at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[?:?]  
> at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616) ~[?:?]  
> at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622) ~[?:?]  
> at java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627) ~[?:?]  
> at org.elasticsearch.node.NodeConstruction.construct(NodeConstruction.java:946) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:295) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.node.Node.(Node.java:201) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.bootstrap.Elasticsearch$2.(Elasticsearch.java:385) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:385) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:97) ~[elasticsearch-8.18.1.jar:?]  
> [2025-06-04T13:21:27,267][ERROR][o.e.b.Elasticsearch] [DevES2] fatal exception while booting Elasticsearch  
> org.elasticsearch.ElasticsearchSecurityException: failed to load SSL configuration [xpack.security.http.ssl] - cannot load PEM certificate\_authorities from [/app/elasticsearch-8.18.1/config/elastic-certificates.p12] due to CertificateParsingException (signed fields invalid)  
> at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSslConfigurations$11(SSLService.java:622) ~[?:?]  
> at java.util.HashMap.forEach(HashMap.java:1429) ~[?:?]  
> at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1707) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLService.loadSslConfigurations(SSLService.java:618) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLService.(SSLService.java:160) ~[?:?]  
> at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:496) ~[?:?]  
> at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:325) ~[?:?]  
> at org.elasticsearch.node.NodeConstruction.lambda$construct$17(NodeConstruction.java:924) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.plugins.PluginsService.lambda$flatMap$0(PluginsService.java:186) ~[elasticsearch-8.18.1.jar:?]  
> at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]  
> at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197) ~[?:?]  
> at java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722) ~[?:?]  
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]  
> at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]  
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575) ~[?:?]  
> at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:260) ~[?:?]  
> at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:616) ~[?:?]  
> at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:622) ~[?:?]  
> at java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:627) ~[?:?]  
> at org.elasticsearch.node.NodeConstruction.construct(NodeConstruction.java:946) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:295) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.node.Node.(Node.java:201) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.bootstrap.Elasticsearch$2.(Elasticsearch.java:385) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:385) ~[elasticsearch-8.18.1.jar:?]  
> at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:97) ~[elasticsearch-8.18.1.jar:?]  
> Caused by: org.elasticsearch.common.ssl.SslConfigException: cannot load PEM certificate\_authorities from [/app/elasticsearch-8.18.1/config/elastic-certificates.p12] due to CertificateParsingException (signed fields invalid)  
> at org.elasticsearch.common.ssl.SslFileUtil.securityException(SslFileUtil.java:125) ~[?:?]  
> at org.elasticsearch.common.ssl.SslFileUtil.securityException(SslFileUtil.java:107) ~[?:?]  
> at org.elasticsearch.common.ssl.PemTrustConfig.readCertificates(PemTrustConfig.java:104) ~[?:?]  
> at org.elasticsearch.common.ssl.PemTrustConfig.createTrustManager(PemTrustConfig.java:80) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:479) ~[?:?]  
> at java.util.HashMap.computeIfAbsent(HashMap.java:1228) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSslConfigurations$11(SSLService.java:620) ~[?:?]  
> ... 24 more  
> Caused by: java.security.cert.CertificateParsingException: signed fields invalid  
> at sun.security.x509.X509CertImpl.parse(X509CertImpl.java:1443) ~[?:?]  
> at sun.security.x509.X509CertImpl.(X509CertImpl.java:143) ~[?:?]  
> at sun.security.x509.X509CertImpl.newX509CertImpl(X509CertImpl.java:253) ~[?:?]  
> at sun.security.provider.X509Factory.parseX509orPKCS7Cert(X509Factory.java:475) ~[?:?]  
> at sun.security.provider.X509Factory.engineGenerateCertificates(X509Factory.java:360) ~[?:?]  
> at java.security.cert.CertificateFactory.generateCertificates(CertificateFactory.java:480) ~[?:?]  
> at org.elasticsearch.common.ssl.PemUtils.readCertificates(PemUtils.java:690) ~[?:?]  
> at org.elasticsearch.common.ssl.PemTrustConfig.readCertificates(PemTrustConfig.java:98) ~[?:?]  
> at org.elasticsearch.common.ssl.PemTrustConfig.createTrustManager(PemTrustConfig.java:80) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:479) ~[?:?]  
> at java.util.HashMap.computeIfAbsent(HashMap.java:1228) ~[?:?]  
> at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSslConfigurations$11(SSLService.java:620) ~[?:?]  
> ... 24 more

Kindly provide the guidance. Appreciate your help!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [June 4, 2025, 1:35pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/10 "2025-06-04T13:35:23Z")

</div>

There is some error in your certificate, not sure exactly what, but this is what elastic is complaining.

I've wrote an article a couple of years ago on how to configure security in a 3 node cluster, it may help you fix the missing pieces, you can find it [here](https://blog.leandrojmp.com/posts/en/2022/10/elasticsearch-security).

---

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 5, 2025, 8:10am UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/11 "2025-06-05T08:10:44Z")

</div>

Great article @leandrojmp . Though I am using tar.gz, I am able to sync with the rpm one. Let me follow the process, try again and keep you posted.

Much appreciated your help on this!

---

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 6, 2025, 8:05am UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/12 "2025-06-06T08:05:32Z")

</div>

Thank you @leandrojmp . I was able to generate the certs and validated them successfully.  
I am waiting for the bootstrap configurations to be completed by the infra admins on the servers. Once done, will validate and confirm the successful launch of the ES instance.

---

<div class="post-metadata">

**Author:** ![IamGuna](https://avatars.discourse-cdn.com/v4/letter/i/41988e/32.png) [@IamGuna](https://discuss.elastic.co/u/IamGuna)\
**Post date:** [June 6, 2025, 1:24pm UTC](https://discuss.elastic.co/t/setting-up-3-node-cluster-for-first-time-and-getting-errors/378775/13 "2025-06-06T13:24:22Z")

</div>

Thank you @leandrojmp . My issue was resolved and I was able to bring up the cluster.

Much appreciated your guidance and continuous support on this.