Configuring Certs Results in "Kibana server is not ready yet."

Elastic Stack Kibana
1

I'm having trouble configuring kibana on a remote fedora38 server. I already have ES working properly, but when I try to connect to the kibana console from my desktop, I receive just "Kibana server is not ready yet."

I had kibana working properly at one point, but now I can't figure out what changed. I'd also like to set it up to use SSL to my browser. I read that xpack requires a license in order to use Let's Encrypt certs - is that correct?

I had included my letsencrypt certs but then I couldn't connect at all, instead of being able to connect with "kibana server is not ready yet." I made sure the private key was readable by the kibana process.

server.ssl.enabled: true
server.ssl.certificate: /etc/letsencrypt/fullchain.pem
server.ssl.key: /etc/letsencrypt/privkey.pem

I'm including my kibana config below.

# cat kibana.yml|grep -Ev '^$|#'
server.host: "209.216.NNN.157"
server.publicBaseUrl: "https://havoc.example.com:5601"
server.name: "havoc.example.com"
elasticsearch.ssl.verificationMode: none
logging:
  appenders:
    file:
      type: file
      fileName: /var/log/kibana/kibana.log
      layout:
        type: json
  root:
    appenders:
      - default
      - file
pid.file: /run/kibana/kibana.pid
elasticsearch.hosts: ['https://havoc.example.com:9200']
elasticsearch.serviceAccountToken: AAEAAWVsYXN0aWMva2liYW...
elasticsearch.ssl.certificateAuthorities: [/var/lib/kibana/ca_1706039228885.crt]
xpack.fleet.outputs: [{id: fleet-default-output, name: default, is_default: true, is_default_monitoring: true, type: elasticsearch, hosts: ['https://havoc.example.com:9200'], ca_trusted_finger
print: 253249c4f3d...}]
xpack.encryptedSavedObjects.encryptionKey: d917dd8375cd6...
xpack.reporting.encryptionKey: 579c02f42151...
xpack.security.encryptionKey: ba29bc7d7bd...
console.ui.enabled: true
csp.strict: false
csp.warnLegacyBrowsers: false

There are also quite a few warnings/errors in the logs. I also tried to set a username and password, but it says it's incompatible with serviceAccountToken?

May 15 22:17:39 havoc kibana[4016589]: [2024-05-15T22:17:39.767-04:00][WARN ][plugins.security.config] Session cookies will be transmitted over insecure connections. This is not recommended.
May 15 22:17:40 havoc kibana[4016589]: [2024-05-15T22:17:40.299-04:00][INFO ][plugins.notifications] Email Service Error: Email connector not specified. {"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2024-05-15T22:17:39.767-04:00","message":"Session cookies will be transmitted over insecure connections. This is not recommended.","log":{"level":"WARN","logger":"plugins.security.config"},"process":{"pid":4016589,"uptime":27.06279409},"trace":"id":"9790cf6840b6bca45954d8a53676bcef"},"transaction":{"id":"ee31d4bf31613c0f"}}
May 15 22:17:51 havoc kibana[4016589]: [2024-05-15T22:17:51.095-04:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. write EPROTO 40E84F155B7F0000:error:0A00010B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:354:

Here is my elasticsearch config:

# cat elasticsearch.yml|grep -Ev '^$|#'
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 209.216.NNN.157
network.bind_host: 209.216.NNN.157
xpack.security.enabled: false
xpack.security.enrollment.enabled: false
xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: false
  verification_mode: certificate
  keystore.path: certs/transport.p12
  truststore.path: certs/transport.p12
cluster.initial_master_nodes: ["havoc.example.com"]
http.host: 0.0.0.0
2

Your configuration is wrong, you configured kibana to talk with Elasticsearch using https, but you explicitly disabled https in your elasticsearch.yml.

In fact, you disabled the security entirely in your cluster.

Keep in mind that exposing your cluster to the internet with security disabled it is just a matter of time until your data is accessed or deleted.

You need to fix your elasticsearch configuration and enable security.

Where you read it? I don't think this is true.