After renewing certificates, Kibana shows multiple expiry dates (2028 vs 2125)

Elastic Stack Elasticsearch
1

We have currently set up 3 Elasticsearch clusters with master nodes on 1.1.1.10, 1.1.1.11, and 1.1.1.12.

Since the certificates had expired, we generated new ones. On the master node (1.1.1.10), we created http.p12, transport.p12, and http_ca.crt, and then copied them to the other nodes (1.1.1.11 and 1.1.1.12).

To verify the certificate validity, we ran the following commands, and they all returned the same output:

echo | openssl s_client -connect 1.1.1.10:9200 -servername 1.1.1.10:9200 2>/dev/null | openssl x509 -noout -dates -subject -issuer 
echo | openssl s_client -connect 1.1.1.11:9200 -servername 1.1.1.11:9200 2>/dev/null | openssl x509 -noout -dates -subject -issuer 
echo | openssl s_client -connect 1.1.1.12:9200 -servername 1.1.1.11:9200 2>/dev/null | openssl x509 -noout -dates -subject -issuer

However, when checking in Kibana with GET _ssl/certificates, we see the following output:

[
  {
    "path": "certs/http.p12",
    "format": "PKCS12",
    "alias": "ca",
    "subject_dn": "CN=Elastic Certificate Tool Autogenerated CA",
    "serial_number": "bd....36", 
    "has_private_key": false,
    "expiry": "2028-09-25T09:18:24.000Z",
    "issuer": "CN=Elastic Certificate Tool Autogenerated CA"
  },
  {
    "path": "certs/http.p12",
    "format": "PKCS12",
    "alias": "instance",
    "subject_dn": "CN=Elastic Certificate Tool Autogenerated CA",
    "serial_number": "bd....36", 
    "has_private_key": false,
    "expiry": "2028-09-25T09:18:24.000Z",
    "issuer": "CN=Elastic Certificate Tool Autogenerated CA"
  },
  {
    "path": "certs/http.p12",
    "format": "PKCS12",
    "alias": "instance",
    "subject_dn": "CN=instance",
    "serial_number": "bf....86", 
    "has_private_key": true,
    "expiry": "2125-09-02T09:18:51.000Z",
    "issuer": "CN=Elastic Certificate Tool Autogenerated CA"
  },
  {
    "path": "certs/transport.p12",
    "format": "PKCS12",
    "alias": "ca",
    "subject_dn": "CN=Elastic Certificate Tool Autogenerated CA",
    "serial_number": "e9....e3", 
    "has_private_key": false,
    "expiry": "2028-09-25T08:43:32.000Z",
    "issuer": "CN=Elastic Certificate Tool Autogenerated CA"
  },
  {
    "path": "certs/transport.p12",
    "format": "PKCS12",
    "alias": "instance",
    "subject_dn": "CN=instance",
    "serial_number": "16....6b", 
    "has_private_key": true,
    "expiry": "2125-09-02T08:43:59.000Z",
    "issuer": "CN=Elastic Certificate Tool Autogenerated CA"
  },
  {
    "path": "certs/transport.p12",
    "format": "PKCS12",
    "alias": "instance",
    "subject_dn": "CN=Elastic Certificate Tool Autogenerated CA",
    "serial_number": "e9....e3", 
    "has_private_key": false,
    "expiry": "2028-09-25T08:43:32.000Z",
    "issuer": "CN=Elastic Certificate Tool Autogenerated CA"
  }
]

We would like to know:

  1. Why do the certificates appear differently in Kibana compared to the OpenSSL command results?

  2. Which certificate expiry dates should we follow for the next renewal? (e.g., 2028 vs 2125)

2

You are actively using two certificates.

  1. http.p12
  2. transport.p12

The http certificate used for HTTP communication which uses port 9200 by default. The other certificate for TRANSPORT communication that is for the Elasticsearch internal node communication which uses port 9300 by default.

You can use the same certificate or different ones for HTTP and TRANSPORT layers.

As a clear answer to your question:

You should take care of both certificate expiry dates.

What happen if your certificate expire?

  • If the transport layer certificate expire, certs/transport.p12, nothing will happen until your nodes restart. When a node restarted it can’t connect back to the cluster.
  • If the http layer certificate expire, and if the client or application connected to the http port with certificate check, the request will fail.

Because with openssl command you are checking HTTP port, 9200. It should show 2125-09-02T09:18:51.000Z as an expiration date.