Generates self-signed client certificates (not server certificates) for Elasticsearch clients

[patpanda]

• What I am trying to achieve

Generates self-signed client certificate (not server certificates) for clients trying to connect to Elasticsearch server.

• What did I try:

I ran this command elasticsearch/bin elasticsearch-certutil ca --pem --out /etc/elasticsearch/certs/ca.zip

which successfully gave me the ca.crt and ca.key files (which I will use in below)

Then, step 2, I ran elasticsearch/bin elasticsearch-certutil cert --out /etc/elasticsearch/certs/elastic.zip --name elasticsearchservercertificates --ca-cert /etc/elasticsearch/certs/ca.crt --ca-key /etc/elasticsearch/certs/ca.key --ip 1.1.1.1 --pem

Step 2 gave me successfully elasticsearch server .crt and .key

Finally, in step 3, I am configuring elastic.yml with:

xpack.security.http.ssl:
  enabled: true
  key: /etc/elasticsearch/certs/elasticsearchservercertificates.kry
  certificate: /etc/elasticsearch/certs/elasticsearchservercertificates.crt
  certificate_authorities: /etc/elasticsearch/certs/ca.crt

With step 1 2 and 3, I can have elasticsearch backend with self-signed certificates.

• Question:

Unfortunately, I am having a hard time generating the client certificates, for kibana, logstash, java clients etc.

Knowing this self signed server side configuration, how to generate the client certificates which will connect to this backend?

[TimV (Tim Vernum)]

A quick note on terminology:
It doesn't look like you're trying to generate "self-signed" client certificates (nor do you have "self-signed" server certificates)

In your first step, when you generate the CA, that's a "self-signed" certificate. It was issued (signed) by itself - if you look at the certificate you will see that the "Issuer" and "Subject" are the same.

But in your second step you provide a CA which is the issuer of the server cert. If you look at the certificate it will have a different "Issuer" and "Subject". That means it's not "self-signed" - the cert didn't sign itself, it was signed by a CA.

You are creating a certificate that is signed by a CA that you control, rather than a commercial CA, but that distinction doesn't really change the nature of the certificate - it just means it uses a CA that won't be trusted by other people.

That matters in practice - it's a real thing to care about because it will affect how clients need to be configured in order to connect successfully - but it's not the same as "self signed".

Not trying to be pedantic, but being clear on language and terminology can help reduce confusion.

---

Knowing this self signed server side configuration, how to generate the client certificates which will connect to this backend?

I would recommend that you have a different CA for your server and client certificates. You don't have to separate them, but there are benefits to doing so, and almost no benefit to having them the same.

Here's what I would do.

1. Create a new client-auth-ca

   elasticsearch/bin elasticsearch-certutil ca --pem \
     --ca-dn "CN=Elasticsearch Client Authentication CA" \
     --out /etc/elasticsearch/certs/client-auth-ca.zip
   

   Unfortunately there's no way to tell certutil what name to use for the files inside the zip, so they will still be called ca/ca.crt and ca/ca.key. You probably want to rename them to client-auth-ca.crt and client-auth-ca.key to be clear. The rest of my instructions will assume that you've done that.

2. Change the certificate_authorities in your elasticsearch.yml to point to the new CA

   xpack.security.http.ssl:
     enabled: true
     key: /etc/elasticsearch/certs/elasticsearchservercertificates.kry
     certificate: /etc/elasticsearch/certs/elasticsearchservercertificates.crt
     certificate_authorities: /etc/elasticsearch/certs/client-auth-ca.crt
   

3. If you want to require that clients use a certificate, then you need to turn that on in your yaml as well

   xpack.security.http.ssl:
     enabled: true
     key: /etc/elasticsearch/certs/elasticsearchservercertificates.kry
     certificate: /etc/elasticsearch/certs/elasticsearchservercertificates.crt
     certificate_authorities: /etc/elasticsearch/certs/client-auth-ca.crt
     client_authentication: required
   

   You can use optional instead of required, but then you will need something else (e.g. a pki realm) to check for client certificates.

4. Issue 1 certificate for each client that you want to use, for example

   elasticsearch/bin elasticsearch-certutil cert --pem \
     --out /etc/elasticsearch/certs/logstash.zip \
     --name logstash \
     --ca-cert /etc/elasticsearch/certs/client-auth-ca.crt \
     --ca-key /etc/elasticsearch/certs/client-auth-ca.key 
   

5. That certificate can be tested in curl with

   curl --cert /etc/elasticsearch/certs/logstash.crt \
     --key /etc/elasticsearch/certs/logstash.key \
     --cacert /etc/elasticsearch/certs/ca.crt \
     https://name.of.your.server:9200/
   

   You may need to also provide a username/password depending on your ES node configuration, but you should see that you can connect with logstash.crt + logstash.key and the handshake fails without it.

There's a blog post that covers some of this here

• Elasticsearch Security: Configure TLS/SSL & PKI Authentication | Elastic Blog

but it's getting a bit old and you may need to change a few things to work with newer versions of ES.