# Unable to connect to kibana with a secure cluster

**URL:** <https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444>\
**Category:** Kibana\
**Tags:** elastic-stack-security\
**Created:** [October 12, 2022, 3:13pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444 "2022-10-12T15:13:09Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 12, 2022, 3:13pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/1 "2022-10-12T15:13:09Z")

</div>

Hello everyone,  
I have configured a cluster of 5 secure nodes with this configuration (adapted on each node 🙂

```auto
xpack.security.http.ssl:
  enabled: true
  keystore.path: /etc/elasticsearch/new/http.p12
  truststore.path: /etc/elasticsearch/new/http.p12
  client_authentication: required
  verification_mode: full
# Enable encryption and mutual authentication between cluster node
xpack.security.transport.ssl:
  enabled: true
  verification_mode: full
  client_authentication: required
  keystore.path: /etc/elasticsearch/new/node-1.p12
  truststore.path: /etc/elasticsearch/new/node-1.p12

```

For the http part I generated one certificate per cluster node (5) with the command :  
`./bin/elasticsearch-certutil http`

But I can't connect to kibana

```auto
server.port: 5601
server.host: "192.168.1.80"
server.publicBaseUrl: "https://192.168.1.80:5601"
server.name: "kibana1"

server.ssl.enabled: true
#Generated with bin/elasticsearch-certutil cert and the same CA as between the elasticsearch cluster nodes
server.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.truststore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.clientAuthentication: required

elasticsearch.hosts: ["https://192.168.1.10:9200", "https://192.168.1.20:9200","https://192.168.1.30:9200", "https://192.168.1.40:9200", "https://192.168.1.50:9200"]
#Generated with bin/elasticsearch-certutil http
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]

```

If someone can help me because I can't find the solution.

Best regards,

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [October 13, 2022, 2:09am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/2 "2022-10-13T02:09:03Z")

</div>

> [@diegz](#):
>
> But I can't connect to kibana

Why, what happens when you try?

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 13, 2022, 8:08am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/3 "2022-10-13T08:08:08Z")

</div>

> [@warkolm](#):
>
> Why, what happens when you try?

Thanks for your answer.  
When I try connect to kibana with my browser nothing happens.  
Port 5601 is open.

Is there a command to test if the connection between kibana and elasticsearch is done correctly?  
What is the curl command to test the cluster status in ssl?

now my cluster does not form after reboot..  
I have this error on my master nodes (2 master and 1 voting\_only)

```auto
org.elasticsearch.transport.NodeNotConnectedException: [node-2][192.168.1.20:9300] Node not connected
        at org.elasticsearch.transport.ClusterConnectionManager.getConnection(ClusterConnectionManager.java:280) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.transport.TransportService.getConnection(TransportService.java:808) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:718) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.start(TransportNodesAction.java:243) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.action.support.nodes.TransportNodesAction.doExecute(TransportNodesAction.java:122) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.action.support.nodes.TransportNodesAction.doExecute(TransportNodesAction.java:39) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:86) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:163) ~[?:?]
        at org.elasticsearch.action.ActionListener$DelegatingFailureActionListener.onResponse(ActionListener.java:245) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeSystemUser(AuthorizationService.java:620) ~[?:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:257) ~[?:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:159) ~[?:?]
        at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:162) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:127) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticateAsync(AuthenticatorChain.java:93) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:171) ~[?:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:155) ~[?:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$apply$0(SecurityActionFilter.java:104) ~[?:?]
        at org.elasticsearch.xpack.core.security.SecurityContext.executeAsInternalUser(SecurityContext.java:121) ~[?:?]
        at org.elasticsearch.xpack.core.security.SecurityContext.executeAsSystemUser(SecurityContext.java:130) ~[?:?]
        at org.elasticsearch.xpack.core.security.SecurityContext.executeAsSystemUser(SecurityContext.java:126) ~[?:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:104) ~[?:?]
        at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:84) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:61) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:186) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:112) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:90) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:380) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.client.internal.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:676) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.client.internal.support.AbstractClient$ClusterAdmin.nodesStats(AbstractClient.java:774) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.InternalClusterInfoService$AsyncRefresh.execute(InternalClusterInfoService.java:184) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.InternalClusterInfoService.refreshAsync(InternalClusterInfoService.java:409) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.InternalClusterInfoService.clusterChanged(InternalClusterInfoService.java:152) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateListener(ClusterApplierService.java:558) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateListeners(ClusterApplierService.java:544) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:504) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:428) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:154) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:710) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:257) ~[elasticsearch-8.4.1.jar:?]
        at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:223) ~[elasticsearch-8.4.1.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]

```

Best regards,

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 13, 2022, 12:19pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/4 "2022-10-13T12:19:45Z")

</div>

About the errors on the cluster, some nodes were locked in a single cluster because I had not removed the option :  
`Configure cluster.initial_master_nodes`  
after the formation of the cluster.

I had to reform the cluster with this procedure:

- Shut down all the nodes.
- Delete each node completely by deleting the content of their data folders. (path.data in elasticsearch.ym)
- Configure cluster.initial\_master\_nodes as described above. (to be deleted after formation of the cluster ! !!!!!)
- Configure discovery.seed\_hosts or discovery.seed\_providers and other relevant - discovery settings.
- Restart all nodes and verify that they have formed a single cluster.

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 13, 2022, 2:46pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/5 "2022-10-13T14:46:56Z")

</div>

Hello @warkolm,  
For Kibana when I test the port 5601, with telnet 192.168.1.80 5601 the connection is refused .  
The port is well opened in the firewall.

Best regards,

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [October 14, 2022, 12:34am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/6 "2022-10-14T00:34:09Z")

</div>

First of all, you need make sure your Elasticsearch cluster is correctly formed and up running. You can test that with curl to check the [Cluster health API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html). Or at least you should not see any errors in the server logs.

Second, your kibana configuration is likely a mismatch with ES configuration. You configured ES to require client\_authentication on HTTP:

```auto
xpack.security.http.ssl:
  enabled: true
  keystore.path: /etc/elasticsearch/new/http.p12
  truststore.path: /etc/elasticsearch/new/http.p12
  client_authentication: required

```

But you did not configure Kibana to send its certificate. I guess you probably misunderstood the following kibana settings

```auto
server.ssl.enabled: true
#Generated with bin/elasticsearch-certutil cert and the same CA as between the elasticsearch cluster nodes
server.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.truststore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.clientAuthentication: required

```

The above settings mandates client\_authentication when a client talks to Kibana, _not_ when Kbiana talks to Elasticsearch. For that you need to use `elasticsearch.ssl.keystore.path` and its friends. Please refer to the [Kibana's settings doc](https://www.elastic.co/guide/en/kibana/master/settings.html) for details.

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 14, 2022, 8:33am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/7 "2022-10-14T08:33:04Z")

</div>

> [@diegz](#):
>
> ```auto
> elasticsearch.hosts: ["https://192.168.1.10:9200", "https://192.168.1.20:9200","https://192.168.1.30:9200", "https://192.168.1.40:9200", "https://192.168.1.50:9200"]
> #Generated with bin/elasticsearch-certutil http
> elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]
> 
> ```

Thank you so much for your answer @Yang_Wang  
I need to keep these settings ;

```auto
server.ssl.enabled: true
---
#Generated with bin/elasticsearch-certutil cert and the same CA as between the elasticsearch cluster nodes
server.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.truststore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.clientAuthentication: required

```

should I add these parameters?

```auto
elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]
elasticsearch.ssl.keystore.path: "/etc/kibana/certs/http.p12"
elasticsearch.ssl.alwaysPresentCertificate: true.
elasticsearch.ssl.verificationMode: full

```

In elasticsearch.ssl.keystore.path I have to add a certificate generated with the certutil http command and fill in a kibana node? or is it the same as in server.ssl.keystore.path?

I don't understand because when generating http certificates an elasticsearch-ca.pem is generated to trust the chain.

Thank you for your precious help

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 14, 2022, 2:42pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/8 "2022-10-14T14:42:51Z")

</div>

With the configuration in my previous post, I get this error message:

```auto
journalctl -u kibana -f
FATAL Error: [config validation of [elasticsearch].ssl.keystore]: could not parse object value from json input

```

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [October 17, 2022, 4:40am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/9 "2022-10-17T04:40:17Z")

</div>

That usually means syntax error in the configuration file. Please double check that. A few things I noticed:

1. You may need configure `elasticsearch.ssl.keystore.password` if the keystore is password protected or empty string if it is not (depending on how it is generated)
2. There is a period after `elasticsearch.ssl.alwaysPresentCertificate: true` in your shareed configuration.

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 17, 2022, 8:00am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/10 "2022-10-17T08:00:34Z")

</div>

> [@Yang\_Wang](#):
>
> elasticsearch.ssl.keystore.password

I managed to correct the previous mistakes.  
Kibana starts correctly but it tells me :  
` Unable to retrieve version information from Elasticsearch nodes`

And when I connect with my browser, I get this message after bypassing the certificate warning:  
SSL\_ERROR\_RX\_CERTIFICATE\_REQUIRED\_ALERT

Thanks again for your help @Yang_Wang

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 19, 2022, 11:26am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/12 "2022-10-19T11:26:10Z")

</div>

The previous errors were related to the keystore password which I solved with the following commands:

```auto
bin/kibana-keystore add server.ssl.truststore.password
bin/kibana-keystore add server.ssl.keystore.password

```

I think that the current error :  
`Unable to retrieve version information from Elasticsearch nodes`  
Is related to the authentication between the elasticsearch cluster and kibana.

Could someone tell me what I need to change in my configuration?  
Example of an elasticsearch node

```auto
xpack.security.http.ssl:
  enabled: true
  keystore.path: /etc/elasticsearch/new/http.p12
  truststore.path: /etc/elasticsearch/new/http.p12
  client_authentication: required
  verification_mode: full
# Enable encryption and mutual authentication between cluster node
xpack.security.transport.ssl:
  enabled: true
  verification_mode: full
  client_authentication: required
  keystore.path: /etc/elasticsearch/new/node-1.p12
  truststore.path: /etc/elasticsearch/new/node-1.p12

```

The kibana configuration :

```auto

server.ssl.enabled: true
server.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.truststore.path: "/etc/kibana/certs/kibana1.p12"
server.ssl.clientAuthentication: required
#server.ssl.certificate: /path/to/your/server.crt
#server.ssl.key: /path/to/your/server.key

# The URLs of the Elasticsearch instances to use for all your queries.
elasticsearch.hosts: ["https://192.168.1.10:9200", "https://192.168.1.20:9200","https://192.168.1.30:9200", "https://192.168.1.40:9200", "https://192.168.1.50:9200"]

elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]
#elasticsearch.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"
elasticsearch.ssl.alwaysPresentCertificate: true
# To disregard the validity of SSL certificates, change this setting's value to 'none'.
elasticsearch.ssl.verificationMode: full

```

Best regards,

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [October 24, 2022, 12:48am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/13 "2022-10-24T00:48:12Z")

</div>

You need share Elasticsearch logs to find out the actual errors.

I suspect you don't want this

```auto
elasticsearch.ssl.verificationMode: full

```

unless you have generated your certificate carefully enough to including SANs for each Elasticsearch node. You can try `elasticsearch.ssl.verificationMode: certificate` instead.

I suspect you don't want this either

```auto
server.ssl.clientAuthentication: required

```

If you plan to access Kibana via just a browser. In general, I suspect you dont' need client authentication for either Elasticsearch or Kibana configurations. You may have some misunderstanding on what it does. Do you have a reason why it is needed?

In your kibana configuration, there is no credentials for Kibana to connect to Elasticsearch. Have you configured that? Either with legacy username/password or a service account token?

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 24, 2022, 2:16pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/14 "2022-10-24T14:16:38Z")

</div>

Hello @Yang_Wang ,  
Thanks for your answers.

For the transport.ssl certificates I generated from this yml file  
instances :

```auto
> - name : "node-1"
> ip :
> - "192.168.1.10"
> dns :
> - "elasticsearch-node1.lab.fr"
> - nom : "node-2"
> ip :
> - "192.168.1.20"
> dns :
> - "elasticsearch-node2.lab.fr"
> - nom : "node-3"
> ip :
> - "192.168.1.30"
> dns :
> - "elasticsearch-node3.lab.fr"
> - nom : "node-4"
> ip :
> - "192.168.1.40"
> dns :
> - "elasticsearch-node4.lab.fr"
> - nom : "node-5"
> ip :
> - "192.168.1.50"
> dns :
> - "elasticsearch-node5.lab.fr"
> - nom : "kibana1"
> ip :
> - "192.168.1.80"
> dns :
> - "kibana1.lab.fr"
> - nom : "kibana2"
> ip :
> - "192.168.1.90"
> dns :
> - "kibana2.lab.fr"
> - nom : "logstash1"
> ip :
> - "192.168.1.60"
> dns :
> - "logstash1.lab.fr"
> - nom : "logstash2"
> ip :
> - "192.168.1.70"
> dns :
> - "logstash2.lab.fr"

```

For http.ssl certificates here is an example of the answer I made to the command  
./bin/elasticsearch-certutil http

```auto
Generate a CSR? [y/N]N

## Do you have an existing Certificate Authority (CA) key-pair that you wish to use to sign your certificate?
Use an existing CA? [y/N]y
## What is the path to your CA?
CA Path: /usr/share/elasticsearch/elastic-stack-ca.p12
Password for elastic-stack-ca.p12:

## How long should your certificates be valid?
For how long should your certificate be valid? [5y]

## Do you wish to generate one certificate per node?
Generate a certificate per node? [y/N]y

## What is the name of node #1?
node #1 name: node-1

## Which hostnames will be used to connect to node-1?

These hostnames will be added as "DNS" names in the "Subject Alternative Name"
(SAN) field in your certificate.

elasticsearch-node1.lab.fr

You entered the following hostnames.

 - elasticsearch-node1.lab.fr

Is this correct [Y/n]y

## Which IP addresses will be used to connect to node-1?

192.168.1.10

You entered the following IP addresses.

 - 192.168.1.10

Is this correct [Y/n]y

## Other certificate options

Key Name: node-1
Subject DN: CN=node-1
Key Size: 2048

```

Is it correct?

```auto
elasticsearch.ssl.verificationMode: full

```

normally if it is the ip is the hostname (dns) is informed the full mode is not a problem no?

```auto
server.ssl.clientAuthentication: required

```

I commented on it, it doesn't seem necessary.

Here are the error feedbacks from elasticsearch

```auto
[2022-10-24T16:12:05,205][WARN][o.e.h.AbstractHttpServerTransport] [node-4] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/192.168.1.40:9200, remoteAddress=/192.168.1.80:35384}
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

```

`Caused by: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors`

```auto
[2022-10-24T16:12:06,290][WARN][o.e.c.s.DiagnosticTrustManager] [node-4] failed to establish trust with client at [<unknown host>]; the client provided a certificate with subject name [CN=kibana1], fingerprint [c9a35b1c12ec10f39e00c9f920613e4be7dd4bf5], no keyUsage and no extendedKeyUsage; the session uses cipher suite [TLS_AES_256_GCM_SHA384] and protocol [TLSv1.3]; the certificate is issued by [CN=Elastic Certificate Tool Autogenerated CA]; the certificate is signed by (subject [CN=Elastic Certificate Tool Autogenerated CA] fingerprint [a6781c1ac84dbe52b929942e418ff19cc42bf8b7]) which is self-issued; the [CN=Elastic Certificate Tool Autogenerated CA] certificate is not trusted in this ssl context ([xpack.security.http.ssl (with trust configuration: StoreTrustConfig{path=/etc/elasticsearch/new/http.p12, password=<non-empty>, type=PKCS12, algorithm=PKIX})]); this ssl context does trust a certificate with subject [CN=Elastic Certificate Tool Autogenerated CA] but the trusted certificate has fingerprint [2d8da61f994c4a727e8378ae4f76433ed014d6e2]
sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

```

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [October 25, 2022, 5:29am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/15 "2022-10-25T05:29:28Z")

</div>

The error message says the server (Elasticsearch) does not trust the client (Kibana)'s certificate. The important bits from the error message are:

> the certificate is signed by (subject [CN=Elastic Certificate Tool Autogenerated CA] fingerprint [a6781c1ac84dbe52b929942e418ff19cc42bf8b7])

> this ssl context does trust a certificate with subject [CN=Elastic Certificate Tool Autogenerated CA] but the trusted certificate has fingerprint [2d8da61f994c4a727e8378ae4f76433ed014d6e2]

The client cert is signed by a CA of the same name but different fingerprint. I suspect there is a mismatch in the generated files. Is it possible that you have ran the `elasticsearch-certutil` multiple times and have used different CA file for generating different files (http vs node certs).

You can check whether it is the case by:

1. Check the issuer signature of Kibana's client cert with `keytool -list -v -keystore /etc/kibana/certs/kibana1.p12`
2. Compare with the trusted CA's signature with `keytool -list -v -keystore /etc/elasticsearch/new/http.p12`

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 25, 2022, 8:38am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/16 "2022-10-25T08:38:19Z")

</div>

Thank you very much @Yang_Wang

I redid all the certificates carefully.  
I have this error now on ES :

```auto
[2022-10-25T10:29:54,405][WARN][o.e.h.AbstractHttpServerTransport] [node-1] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/192.168.1.10:9200, remoteAddress=/192.168.1.80:34090}
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty client certificate chain
Caused by: javax.net.ssl.SSLHandshakeException: Empty client certificate chain

```

kibana.yml

```auto

elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/certs/elasticsearch-ca.pem"]
elasticsearch.ssl.verificationMode: full

```

elasticsearch.yml

```auto
xpack.security.http.ssl:
  enabled: true
  keystore.path: /etc/elasticsearch/new/http.p12
  truststore.path: /etc/elasticsearch/new/http.p12
  client_authentication: required
  verification_mode: full

```

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [October 25, 2022, 9:59am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/17 "2022-10-25T09:59:29Z")

</div>

> [@diegz](#):
>
> `Empty client certificate chain`

The error means Kibana didn't send any client certificate. You need something like in your Kibana configuration file:

```auto
elasticsearch.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"

```

You will also need configure the above keystore password with

```auto
bin/kibana-keystore add elasticsearch.ssl.keystore.password

```

Also, have you configured the user (or service token) for Kibana to connect to Elasticsearch? This should be either `elasticsearch.username` and `elasticsearch.password` or `elasticsearch. serviceAccountToken`. You can refer to [this page](https://www.elastic.co/guide/en/kibana/current/settings.html) for details.

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 25, 2022, 2:01pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/18 "2022-10-25T14:01:40Z")

</div>

Thanks @Yang_Wang

I create a service token on es with command :  
`bin/elasticsearch-service-tokens create elastic/kibana my-token`

And I add token in kibana :  
`elasticsearch.serviceAccountToken: "AAEAAWVsYXN0aWMva2liYW5hL215LXRva2VuOlNUWjNpVUYxUZNXNnFIeWdsSll5XUE"`

I change rights in /etc/elasticsearch/service\_tokens

```auto
chown elasticsearch;elasticsearch etc/elasticsearch/service_tokens
chmod 0644 /etc/elasticsearch/service_tokens

```

I restart elasticsearch.service on all nodes

I think I have no more errors about "Empty client certificate chain" in ES after add :  
`elasticsearch.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"`

how to configure the token service for all the nodes of the cluster?

I have this errors now on kibana :

```auto
ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. security_exception: [index_not_found_exception] Reason: no such index [.security]
Error: Failure installing common resources shared between all indices. Server is stopping; must stop all async operations
[ERROR][plugins.ruleRegistry] Error: Failure installing common resources shared between all indices. Server is stopping; must stop all async operations

```

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 25, 2022, 3:16pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/19 "2022-10-25T15:16:16Z")

</div>

I copied the service\_tokens file on all nodes and the connection works!  
I want to say thank you for your help!  
Thanks to you I will be able to put the secure stack into production.  
I am not safe from asking questions when I will have to secure logstash and others.

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 26, 2022, 10:02am UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/20 "2022-10-26T10:02:03Z")

</div>

Hello @Yang_Wang,  
I access the kibana login page but when I want to log in username or password is incorrect.  
I use this command to fix elastic password :  
`bin/elasticsearch-reset-password -i -u elastic`

In the elastic logs I always get the error when I try to log in Kibana.

```auto
[WARN][o.e.h.AbstractHttpServerTransport] [node-1] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/192.168.1.10:9200, remoteAddress=/192.168.1.80:38968}
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Empty client certificate chain

```

yet I added as you told me  
`elasticsearch.ssl.keystore.path: "/etc/kibana/certs/kibana1.p12"`

Why ?

Thank you again for your help

---

<div class="post-metadata">

**Author:** ![diegz](https://avatars.discourse-cdn.com/v4/letter/d/d07c76/32.png) [@diegz](https://discuss.elastic.co/u/diegz)\
**Post date:** [October 26, 2022, 12:15pm UTC](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444/21 "2022-10-26T12:15:22Z")

</div>

I solved my authentication problem by adding the :  
`elasticsearch.ssl.alwaysPresentCertificate: true`

[Next page](https://discuss.elastic.co/t/unable-to-connect-to-kibana-with-a-secure-cluster/316444.md?page=2)