After setting PKI realm on ElasticSearch unable to login to Kibana

Elastic Stack Kibana
1

I've setup PKI realm on ElasticSearch but after that I'm not able to login to Kibana. It shows the following message:
"Login is currently disabled. Administrators should consult the Kibana logs for more details."

log [19:38:59.040] [info][listening] Server running at https://localhost:5601
log [19:38:59.045] [server][uuid][uuid] Resuming persistent Kibana instance UUID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx
log [19:38:59.050] [error][admin][elasticsearch] Request error, retrying
HEAD https://localhost:9200/ => self signed certificate in certificate chain
log [19:38:59.067] [error][admin][elasticsearch] Request error, retrying
HEAD https://localhost:9200/ => self signed certificate in certificate chain
log [19:38:59.073] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/
log [19:38:59.074] [warning][admin][elasticsearch] No living connections
log [19:38:59.077] [error][status][plugin:xpack_main@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.079] [error][status][plugin:searchprofiler@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.080] [error][status][plugin:ml@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.081] [error][status][plugin:tilemap@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.082] [error][status][plugin:watcher@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.083] [error][status][plugin:index_management@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.086] [error][status][plugin:graph@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.087] [error][status][plugin:security@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.088] [error][status][plugin:grokdebugger@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.091] [error][status][plugin:logstash@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.092] [error][status][plugin:reporting@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.094] [error][status][plugin:elasticsearch@6.3.2] Status changed from yellow to red - Unable to connect to Elasticsearch at https://localhost:9200.
log [19:38:59.096] [debug][license][xpack] Calling [data] Elasticsearch _xpack API. Polling frequency: 30001
log [19:38:59.102] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/
log [19:38:59.104] [warning][admin][elasticsearch] No living connections
log [19:38:59.182] [error][data][elasticsearch] Request error, retrying
GET https://localhost:9200/_xpack => self signed certificate in certificate chain
log [19:38:59.223] [warning][data][elasticsearch] Unable to revive connection: https://localhost:9200/
log [19:38:59.228] [warning][data][elasticsearch] No living connections
log [19:38:59.231] [warning][license][xpack] License information from the X-Pack plugin could not be obtained from Elasticsearch for the [data] cluster. Error: No Living connections
log [19:38:59.236] [error][status][plugin:xpack_main@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.242] [error][status][plugin:searchprofiler@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.247] [error][status][plugin:ml@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.251] [error][status][plugin:tilemap@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.258] [error][status][plugin:watcher@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.264] [error][status][plugin:index_management@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.269] [error][status][plugin:graph@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.272] [error][status][plugin:security@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.280] [error][status][plugin:grokdebugger@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.283] [error][status][plugin:logstash@6.3.2] Status changed from red to red - No Living connections
log [19:38:59.286] [error][status][plugin:reporting@6.3.2] Status changed from red to red - No Living connections
log [19:39:01.635] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/
log [19:39:01.639] [warning][admin][elasticsearch] No living connections
log [19:39:01.989] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/
log [19:39:01.992] [warning][admin][elasticsearch] No living connections
..
..
..

error [19:40:05.629] [error][client][connection] Error: socket hang up
at TLSSocket. (_tls_wrap.js:876:25)
at emitOne (events.js:121:20)
at TLSSocket.emit (events.js:211:7)
at _handle.close (net.js:557:12)
at Socket.done (_tls_wrap.js:356:7)
at Object.onceWrapper (events.js:315:30)
at emitOne (events.js:121:20)
at Socket.emit (events.js:211:7)
at TCP._handle.close [as _onclose] (net.js:557:12)
log [19:40:05.925] [debug][connection] 101057795:error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request:openssl\ssl\s23_srvr.c:397:

log [19:40:05.937] [debug][connection] 101057795:error:1407609C:SSL routines:SSL23_GET_CLIENT_HELLO:http request:openssl\ssl\s23_srvr.c:397:

ops [19:40:07.938] memory: 107.3MB uptime: 0:01:18 load: [0.00 0.00 0.00] delay: 8.133
log [19:40:08.023] [warning][admin][elasticsearch] Unable to revive connection: https://localhost:9200/
log [19:40:08.078] [warning][admin][elasticsearch] No living connections

2

Please provide details about how you setup the PKI Realm.
It is difficult to diagnose problems if we don't actually know what changes you made.

3

elasticsearch.yml

bootstrap.memory_lock: false
cluster.name: elasticsearch
http.port: 9200
node.data: true
node.ingest: true
node.master: true
node.max_local_storage_nodes: 1
node.name: DESKTOP-XXXX
path.data: C:\ProgramData\Elastic\Elasticsearch\data
path.logs: C:\ProgramData\Elastic\Elasticsearch\logs
transport.tcp.port: 9300
#certificate_authorities: ["C:\\ProgramData\\Elastic\\Elasticsearch\\config\\certs\\elastic-stack-ca.p12"]
xpack.license.self_generated.type: basic
xpack:
  ssl:
    keystore:
      type: PKCS12
      path: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.p12
    client_authentication: optional
    verification_mode: certificate
    truststore:
      path: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.p12
      type: PKCS12
    # certificate_authorities: ["C:\\ProgramData\\Elastic\\Elasticsearch\\config\\certs\\elastic-stack-ca.p12"]
  security:
    enabled: true
    transport:
      ssl:
        enabled: true
        verification_mode: certificate
        keystore:
          type: PKCS12
          path: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.p12
        truststore:
          path: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.p12
          type: PKCS12
        client_authentication: optional
        # certificate_authorities: ["C:\\ProgramData\\Elastic\\Elasticsearch\\config\\certs\\elastic-stack-ca.p12"]
    http:
      ssl:
        enabled: true
        verification_mode: certificate
        keystore:
          type: PKCS12
          path: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.p12
        truststore:
          path: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.p12
          type: PKCS12
        client_authentication: optional
        # certificate_authorities: ["C:\\ProgramData\\Elastic\\Elasticsearch\\config\\certs\\elastic-stack-ca.p12"]
    authc:
      realms:
        pki1:
          type: pki
          order: 0
          enabled: true
          # certificate_authorities: ["C:\\ProgramData\\Elastic\\Elasticsearch\\config\\certs\\elastic-stack-ca.p12"]
4

kibana.yml

elasticsearch.username: "kibana"
elasticsearch.password: "XXXXX"
server.ssl.enabled: true
server.ssl.certificate: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.crt
server.ssl.key: C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-certificates.key
server.ssl.keyPassphrase: XXXXX
elasticsearch.ssl.certificateAuthorities: [ "C:\ProgramData\Elastic\Elasticsearch\config\certs\elastic-stack-ca.p12" ]
elasticsearch.ssl.verificationMode: certificate
elasticsearch.requestTimeout: 120000
logging.verbose: true

5

Used elasticsearch-certutil to generate the certificates.

Used the following to generate .crt and .key files out of the .p12 file

openssl pkcs12 -in filename.pfx -nocerts -out filename.key

openssl pkcs12 -in filename.pfx -clcerts -nokeys -out filename.crt
6

Kibana does not support PKCS12 files in the certificateAuthorities list.
I'm afraid you'll need to export the CA cert as a PEM file using openssl.

7

Oh! Sorry for missing the indentation. I didnt notice it. Thanks for your update.

8

OK. So do we need to have separate .crt and .key files out of the CA .p12 file? Or do we need certificate and key in the same .pem file?

9

Thanks! I just used the .crt file and yes it worked.

However, I also am noticing this error message often usually after I have triggered a query from kibana.

06:59:43.817] [debug][plugin] Checking Elasticsearch version
error [06:59:45.558] [error][client][connection] Error: socket hang up
at TLSSocket. (_tls_wrap.js:876:25)
at emitOne (events.js:121:20)
at TLSSocket.emit (events.js:211:7)
at _handle.close (net.js:557:12)
at Socket.done (_tls_wrap.js:356:7)
at Object.onceWrapper (events.js:315:30)
at emitOne (events.js:121:20)
at Socket.emit (events.js:211:7)
at TCP._handle.close [as _onclose] (net.js:557:12)
log [06:59:46.236] [debug][license][xpack] Calling [data] Elasticsearch _xpack API. Polling frequency: 30001
log [06:59:46.353] [debug][plugin] Checking Elasticsearch version
ops [06:59:47.878] memory: 116.7MB uptime: 0:08:27 load: [0.00 0.00 0.00] delay: 6.641
log [06:59:47.884] [debug][kibana-monitoring][monitoring-ui] Received Kibana Ops event data
log [06:59:48.882] [debug][plugin] Checking Elasticsearch version
log [06:59:51.420] [debug][plugin] Checking Elasticsearch version
log [06:59:52.867] [debug][kibana-monitoring][monitoring-ui] Fetching data from kibana collector
log [06:59:52.873] [debug][kibana-monitoring][monitoring-ui] Fetching data from kibana_stats collector
log [06:59:52.885] [debug][kibana-monitoring][monitoring-ui] Fetching data from kibana_settings collector
log [06:59:52.890] [debug][kibana-monitoring][monitoring-ui] Fetching data from reporting_stats collector
ops [06:59:52.920] memory: 116.8MB uptime: 0:08:32 load: [0.00 0.00 0.00] delay: 17.259
log [06:59:52.931] [debug][kibana-monitoring][monitoring-ui] Received Kibana Ops event data
log [06:59:52.937] [debug][kibana-monitoring][monit

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.