# Kibana 6.5.3 javax.net.ssl.SSLHandshakeException: null cert chain

**URL:** <https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375>\
**Category:** Kibana\
**Created:** [December 18, 2018, 4:39pm UTC](https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375 "2018-12-18T16:39:57Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![totsugeki](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@totsugeki](https://discuss.elastic.co/u/totsugeki)\
**Post date:** [December 18, 2018, 4:39pm UTC](https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375/1 "2018-12-18T16:39:57Z")

</div>

Hello everyone.

I have an elastic cluster in 6.2.4, with the x-pack plugin, a licence, and full ssl required (http, transports)  
A kibana is also installed and is authenticated with signed certificates

I am trying to install the latest version 6.5.3.  
Everything works fine for Elastic, but no way to connect Kibana with ssl verification.

When I try to access kibana i get :  
`{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred"}`

Kibana logs :

```auto
{"type":"log","@timestamp":"2018-12-18T16:16:42Z","tags":["info","authentication"],"pid":11074,"message":"Authentication attempt failed: No Living connections"}
{"type":"error","@timestamp":"2018-12-18T16:16:41Z","tags":[],"pid":11074,"level":"error","error":{"message":"No Living connections","name":"Error","stack":"Error: No Living connections\n at sendReqWithConnection (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:226:15)\n at next (/usr/share/kibana/node_modules/elasticsearch/src/lib/connection_pool.js:214:7)\n at _combinedTickCallback (internal/process/next_tick.js:132:7)\n at process._tickDomainCallback (internal/process/next_tick.js:219:9)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":"","query":{},"pathname":"/app/infra","path":"/app/infra","href":"/app/infra"},"message":"No Living connections"}
{"type":"response","@timestamp":"2018-12-18T16:16:41Z","tags":[],"pid":11074,"method":"get","statusCode":500,"req":{"url":"/app/infra","method":"get","headers":{"host":"node-01","user-agent":"Mozilla/5.0 (Windows NT 6.1; rv:52.0) Gecko/20100101 20000523 Firefox/52.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate, br","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.10.10.1","userAgent":"10.10.10.5"},"res":{"statusCode":500,"responseTime":26,"contentLength":9},"message":"GET /app/infra 500 26ms - 9.0B"}

```

Elastic logs:

```auto
[2018-12-18T17:19:55,408][WARN][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node-01] caught exception while handling client http traffic, closing connection [id: 0x8426cb88, L:0.0.0.0/0.0.0.0:9200 ! R:/10.10.10.1:42482]
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: null cert chain
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:472) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
.....
Caused by: javax.net.ssl.SSLHandshakeException: null cert chain
        at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[?:?]
        at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[?:?]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:330) ~[?:?]
        at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:318) ~[?:?]
        at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1935) ~[?:?]
        at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:237) ~[?:?]
        at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052) ~[?:?]
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:992) ~[?:?]
        at sun.security.ssl.Handshaker$1.run(Handshaker.java:989) ~[?:?]
        at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_181]
        at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1467) ~[?:?]
        at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1457) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1365) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1199) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1243) ~[netty-handler-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:502) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:441) ~[netty-codec-4.1.30.Final.jar:4.1.30.Final]
        ... 15 more

```

I can curl, using my certs :

```auto
curl --user kibana:password --cacert /etc/kibana/ssl/ca/ca.crt --cert /etc/kibana/ssl/kibana/kibana.crt --key /etc/kibana/ssl/kibana/kibana.key https://node-01:9200/_cat/nodes
10.10.10.100 3 64 3 0.92 0.70 0.54 mdi * node-01

```

In Kibana config :

```auto
server.ssl.enabled: true
server.ssl.certificate: /etc/kibana/ssl/kibana/kibana.crt
server.ssl.key: /etc/kibana/ssl/kibana/kibana.key

elasticsearch.ssl.certificate: /etc/kibana/ssl/kibana/kibana.crt
elasticsearch.ssl.key: /etc/kibana/ssl/kibana/kibana.key

elasticsearch.ssl.certificateAuthorities: ["/etc/kibana/ssl/ca/ca.crt"]

elasticsearch.ssl.verificationMode: full

```

In Elastic config :

```auto
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.client_authentication: required
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/ssl/elasticsearch/elastic.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/ssl/elasticsearch/elastic.crt
xpack.security.transport.ssl.certificate_authorities: ["/etc/elasticsearch/ssl/ca/ca.crt"]

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.client_authentication: required
xpack.security.http.ssl.key: /etc/elasticsearch/ssl/elasticsearch/elastic.key
xpack.security.http.ssl.certificate: /etc/elasticsearch/ssl/elasticsearch/elastic.crt
xpack.security.http.ssl.certificate_authorities: ["/etc/elasticsearch/ssl/ca/ca.crt"]

```

If I disable **_xpack.security.http.ssl.client\_authentication_** and set it to _optional_ or _none_, kibana is working correctly. But i don't want to disable it.

It's strange as I am using the same certificates from before and the curl is working with thoses certs..

Do you have any idea how to fix this ?

Thank you very much.

---

<div class="post-metadata">

**Author:** ![rashmi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rashmi/32/16391_2.png) [@rashmi](https://discuss.elastic.co/u/rashmi)\
**Post date:** [December 19, 2018, 10:41pm UTC](https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375/2 "2018-12-19T22:41:59Z")

</div>

@Larry_Gregory/ @Brandon_Kobel - our security people can help here when they get some time.

Cheers  
Rashmi

---

<div class="post-metadata">

**Author:** ![Brandon\_Kobel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_kobel/32/14829_2.png) [@Brandon\_Kobel](https://discuss.elastic.co/u/Brandon_Kobel)\
**Post date:** [December 19, 2018, 10:58pm UTC](https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375/3 "2018-12-19T22:58:40Z")

</div>

Hey @totsugeki, 6.4 introduced a change in behavior when Kibana send it's certificates for certain ES connections. [https://github.com/elastic/kibana/issues/22643](https://github.com/elastic/kibana/issues/22643) provides some context for what you're experiencing, and as of 6.5 you can use the `elasticsearch.ssl.alwaysPresentCertificate: true` setting in your kibana.yml to always present the certificates to ES; however, care should be used when setting this as if you're using the PKI realm in ES to authenticate users, in certain configurations this can lead to requests from your end-user being authenticated as the internal Kibana system user.

---

<div class="post-metadata">

**Author:** ![totsugeki](https://avatars.discourse-cdn.com/v4/letter/t/b782af/32.png) [@totsugeki](https://discuss.elastic.co/u/totsugeki)\
**Post date:** [December 21, 2018, 9:44am UTC](https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375/4 "2018-12-21T09:44:13Z")

</div>

Than you @Brandon_Kobel.

I missed this change in behavior (where could I have read about this ?)  
So if I understand correctly :

If we set `xpack.security.http.ssl.client_authentication: required` in elasticsearch, it is impossible to use kibana without `elasticsearch.ssl.alwaysPresentCertificate: true` because elasticsearch is expecting a client certificat.

> in certain configurations this can lead to requests from your end-user being authenticated as the internal Kibana system user.

Do you have more informations about this ?  
What are the security implication ?

Thank you very much !

---

<div class="post-metadata">

**Author:** ![Brandon\_Kobel](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_kobel/32/14829_2.png) [@Brandon\_Kobel](https://discuss.elastic.co/u/Brandon_Kobel)\
**Post date:** [December 21, 2018, 3:04pm UTC](https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375/5 "2018-12-21T15:04:28Z")

</div>

> Do you have more informations about this ?  
> What are the security implication?

This is only an issue when using the [PKI Realm](https://www.elastic.co/guide/en/elastic-stack-overview/current/pki-realm.html) in Elasticsearch, and the certificate which Kibana presents to Elasticsearch is used to authenticate the Kibana server's internal user. There are two types of requests which Kibana makes to Elasticsearch, those which it intends to make as Kibana server's internal user (by default the `kibana` user`), and those that it proxies to Elasticsearch on behalf of the end-user which we intend to make using the end-users credentials. On these proxied requests, if we're presenting the certificate and the end-user's certificates, we have to have the realms in Elasticsearch ordered in a specific way so that the end-users credentials take precedent over the certificate, or else we risk these proxied requests being executing on behalf of the Kibana server internal user.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 18, 2019, 3:04pm UTC](https://discuss.elastic.co/t/kibana-6-5-3-javax-net-ssl-sslhandshakeexception-null-cert-chain/161375/6 "2019-01-18T15:04:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
