Elasticsearch node http certificate expiration error

Elastic Stack Elasticsearch
1

While i try to connects the kibana to elasticsearch i get the following error in the kibana's logs and when i try to load the kibana i get the Kibana server is not ready yet. :

[ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. certificate has expired

but the when i check the certificate it has not expired yet. Here is my elasticsearch.yml file

xpack.security.http.ssl:
  enabled: true
  verification_mode: full
  client_authentication: required
  keystore.path: certs/http.keystore.p12
  truststore.path: certs/truststore.p12

and here is the kibana.yml

elasticsearch.hosts: ['https://192.168.7.153:50002']

# kibana to cluster 
elasticsearch.ssl.truststore.path: /etc/kibana/truststore.p12
elasticsearch.ssl.keystore.path: /etc/kibana/cluster.keystore.p12
elasticsearch.ssl.alwaysPresentCertificate: true
elasticsearch.ssl.verificationMode: full

and when i check my cert expiration date with the following openssl command:

  • first extracting cert
    openssl pkcs12 -in http.keystore.p12 -clcerts -nokeys -out del-certificate.crt
  • then checking it expiration date
    openssl x509 -in ./del-certificate.crt -noout -enddate
    i get the
    notAfter=Dec 9 08:06:40 2025 GMT

which shows the certificate is not expired. when in kibana.yml file i change the

elasticsearch.ssl.verificationMode: full
to
elasticsearch.ssl.verificationMode: none

the problem goes away which indicate the problem probably must be with the http certificate provided by elasticsearch to kibana. Is there any tips and suggestion how to troubleshoot? by the way i am trying to have mutalTLS enabled between kibana and cluster.

2

A couple of suggestions to start with:

  1. Try using openssl s_client -showcerts -connect 192.168.7.153:50002 to double check that ES is presenting the certificate that you think it should be.

  2. Double check the date/time on your Kibana server

3

first of all thanks for the reply.

I had checked system time and it was ok, i checked it again and it is still ok on both elasticsearch and kibana mahcine.

about checking the certificate that elasticsearch represent to kibana as you told i did the following:

  1. first i ran the command openssl s_client -showcerts -connect 192.168.7.153:50002 and i got

    image
    image853×562 99 KB

  2. then as my xpack.security.http.ssl.keystore.path refere to
    certs/http.keystore.p12 i ran the command openssl pkcs12 -in http.keystore.p12 -nokeys -clcerts -nodes and i got

    image
    image738×503 87.8 KB

so as the both outputs are the same, i guess it is safe to assume the elasticsearch is representing the intended certificate? and also i guess the second line of the first picture is kind of a proof that the certificate is not expired?

4

UPDATE:
surprisingly (at least for me :slight_smile: ) i have a logstash config as follow:

output {
  elasticsearch {
    hosts => ["https://192.168.7.153:50002"]
    user => "elastic"
    password => "a_pass"
    index => "test_index"
    ssl_truststore_path => "/etc/logstash/certs/truststore.p12"
    ssl_truststore_password => "a_[ass"
    ssl_keystore_path => "/etc/logstash/certs/logst ash-es.keystore.p12"
    ssl_keystore_password =>  "a_pass"
    ssl_enabled => true
    ssl_verification_mode => full
  }
}

and even with ssl_verification_mode => full it is working fine.

5

I'm in guesswork territory now, but...

Maybe the signing certificate has expired?

It looks like your Elasticsearch is configured to send only the leaf certificate (which is normal), so Kibana is relying on the CA from /etc/kibana/truststore.p12

Has that CA expired?

6

That was not the problem. i even tried regenerating

xpack.security.http.ssl.keystore.path: certs/http.keystore.p12

but that did not solve the problem too. I would try again solving the problem in the coming days and if i came across any solution i would post it here, meanwhile any more tips would be appreciated.