Christopher, yes, Kibana was working before over SSL without any issue. It is not running behind a proxy.
The ES certificate is fresh too, but I don't think this is the problem. I tried elasticsearch.ssl.verificationMode: none and I get the same error. I don't think it is from elasticsearch SSL because if I configure server.ssl.enabled: false (which is the Kibana SSL), the error disappear.
I am sure the certificate is valid because in the browser I see the certificate start date being a few days ago, it is reported as being valid, and the Kibana server was installed a few months back, so It is clear that something happened when the new certificate was issued and installed by Letsencrypt.
I tried restarting the services (elasticsearch and kibana) also, I tried restarting the server all together.