TLS Between Elastic & Kibana...Certs Not Working

[Webtrend_Inc]

We have established TLS on our cluster (v7.7.1) by generating self-signed certificates as described in this document. Entire cluster seems to be working fine.

However, to establish TLS between Kibana (residing on a separate server not a part of the cluster) we have made multiple efforts without any success.

We have tried setting

elasticsearch.ssl.truststore.path: "/path/to/elasticsearch-ca.p12"

and tried both the public certificate and node certificate in PKCS format. The cert does not have any encryption password so we have not defined elasticsearch.ssl.truststore.password. When we run kibana, we get following error

 PKCS#12 MAC could not be verified. Invalid password?

We tried converting the pk12 to pem using openssl

 openssl pkcs12 -in elastic-certificates.p12 -cacerts -nokeys -out elasticsearch-ca.pem

and then set the following parameter in Kibana

 elasticsearch.ssl.certificateAuthorities: ["/path/to/elasticsearch-ca.pem"]

In this case, we see following error

  Hostname/IP does not match certificate's altnames: IP: 10.0.x.xxx is not in the cert's list:

Any suggestions on how to resolve this problem?

[Webtrend_Inc]

bumpity bump

Anyone?

[ikakavas (Ioannis Kakavas)]

You need to

• Either re-issue your certificate so that it contains its hostname in the list of the SANs, so that anything( Kibana in this case) can perform hostname validation as it should as part of the TLS handshake
• or ( and I certainly discourage you to do so) set elasticsearch.ssl.verificationMode: certificate in your kibana settings so that it won't try to perform hostname validation and you have a working ( but less secure setup )

[Webtrend_Inc]

Thanks a lot for your response.
I went with the second option and set

elasticsearch.ssl.verificationMode: certificate
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/elasticsearch-ca.pem" ]

But now I get this error

GET https://10.0.xx.xx:9200/_xpack => write EPROTO 140246428264256:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:332:

[ikakavas (Ioannis Kakavas)]

This means that you haven't configured elasticsearch for TLS and it's waiting for plain http connections but you try to access it with https

[Webtrend_Inc]

Yes, you are correct. I had commented out http TLS configuration on my client node. After enabling, Kibana starts without any error but still no joy

The alerting section in kibana still says "You must enable Transport Layer Security
Alerting relies on API keys, which require TLS between Elasticsearch and Kibana. Learn how to enable TLS."

I believe I took all the relevant steps to secure the connection between Kibana and Elasticsearch. Here are the config parameters currently set on my yml file

elasticsearch.hosts: ["https://10.0.xx.xx:9200"]
server.ssl.enabled: true
server.ssl.certificate: /etc/letsencrypt/live/xxx/cert.pem
server.ssl.key: /etc/letsencrypt/live/xxxx/privkey.pem
elasticsearch.ssl.certificateAuthorities: [ "/etc/kibana/certs/elasticsearch-ca.pem" ]
elasticsearch.ssl.verificationMode: certificate
xpack.encryptedSavedObjects.encryptionKey: "some random key"
xpack.security.encryptionKey: "some random key"

What am I missing now?
During kibana startup I see this:

 X-Pack Monitoring Cluster Alerts will not be available: undefined

What does it mean?
Thanks again.

[ikakavas (Ioannis Kakavas)]

What is your elasticsearch config ? Are you certain that TLS for http is enabled in elasticsearch ? The error message says its not.