Certificate hell :)

[vincentv]

Hi all,

i currently have a 3 node ES cluster (6x) with kibana in front, and 3 logstash nodes behind.
Because we use SSO we have TLS/SSL enabled in ES both for the nodes 9300 and clients 9200.

The node to node is working great with certutil generated keystore and truststore, however the SSL cert for the client side is not working.

The client side cert is a wildcard from digicert that matches the loadbalancer address we use for all ES nodes. The Load balancer also has a copy of that key to serve SSL if we so desire.

Kibana and Logstash both were complaining that the certificate authority is not valid.

We were able to get kibana working by disabling certificate verification, however Logstash doesn't seem to want to connect no matter what we do.

We can curl to the loadbalancer with SSL and it works fine, however if we curl directly to the ES host on 9200 it always fails certificate validation.

It seems to me that the Logstash to ELB certs are fine but the one on the ES server itself will not pass validation.

Any assistance to get this working would be appreciated.

NOTE: This wildcard cert, is in fact valid and is being used in production.

[ikakavas (Ioannis Kakavas)]

Hi Vincent,

Side note A: For load balancing in front of ES you can look into using a coordinating node instead of an external LB.

Side note B: If your Elasticsearch instances are not meant to be accessed by clients, but rather only via Kibana, then you could use self-signed certificates for the http layer also ( As you've seen from the errors you're getting, Kibana and Logstash communicate with Elasticsearch over the http layer )

To your issue now:

We were able to get kibana working by disabling certificate verification

That's ok for a workaround now, but you probably want to solve this in a more secure way for production.

The Load balancer also has a copy of that key to serve SSL if we so desire.

Do you want to terminate TLS on the loadbalancer ?

We can curl to the loadbalancer with SSL and it works fine, however if we curl directly to the ES host on 9200 it always fails certificate validation.

Have you configured Elasticsearch to use this keypair for TLS on the http layer ? Certificate validation error seems appropriate here since you're serving a certificate for *.fqdnA.com (LB) from *.fqdnB.com (ES)
Is kibana pointed to the load balancer or directly to an elasticsearch node ?