X-pack and kibana: Client request error: unable to verify the first certificate

Elastic Stack Kibana
1

Hi,

I am evaluating the use of Elasticsearch, Kibana and x-pack 5 and downloaded them, then I followed

https://www.elastic.co/guide/en/x-pack/current/security-getting-started.html#enable-message-authentication
https://www.elastic.co/guide/en/kibana/current/production.html
https://www.elastic.co/guide/en/x-pack/current/ssl-tls.html

I created the self-signed certificate with certgen and setup ssl.

I can login to Kibana through https://x.x.x.x:5601 (IP replaced with x.x.x.x) in a browser.

When I goto "Dev Tools" and try:

    GET _search
    {
    "query": {
        "match_all": {}
    }
    }

I get an:

    Error connecting to 'https://x.x.x.x:9200/_search':

    Client request error: unable to verify the first certificate

But I set elasticsearch.ssl.verify: false in kibana.yml

I do not see any problems for this in the Kibana startup logs (IP replaced with x.x.x.x):

log [19:46:05.092] [info][status][plugin:kibana@5.0.0] Status changed from uninitialized to green - Ready log [19:46:05.123] [info][status][plugin:elasticsearch@5.0.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [19:46:05.130] [info][status][plugin:xpack_main@5.0.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [19:46:05.285] [info][status][plugin:graph@5.0.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [19:46:05.296] [info][status][plugin:monitoring@5.0.0] Status changed from uninitialized to green - Ready log [19:46:05.299] [warning][reporting] Generating a random key for xpack.reporting.encryptionKey. To prevent pending reports from failing on restart, please set xpack.reporting.encryptionKey in kibana.yml log [19:46:05.303] [info][status][plugin:reporting@5.0.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [19:46:05.332] [info][status][plugin:security@5.0.0] Status changed from uninitialized to yellow - Waiting for Elasticsearch log [19:46:05.375] [info][status][plugin:elasticsearch@5.0.0] Status changed from yellow to green - Kibana index ready log [19:46:05.391] [info][license][xpack] Imported license information from Elasticsearch: mode: trial | status: active | expiry date: 2016-12-01T16:27:35-06:00 log [19:46:05.410] [info][status][plugin:monitoring@5.0.0] Status changed from green to yellow - Waiting for Monitoring Health Check log [19:46:05.411] [info][status][plugin:xpack_main@5.0.0] Status changed from yellow to green - Ready log [19:46:05.412] [info][status][plugin:graph@5.0.0] Status changed from yellow to green - Ready log [19:46:05.414] [info][status][plugin:reporting@5.0.0] Status changed from yellow to green - Ready log [19:46:05.415] [info][status][plugin:security@5.0.0] Status changed from yellow to green - Ready log [19:46:05.422] [info][status][plugin:console@5.0.0] Status changed from uninitialized to green - Ready log [19:46:06.970] [info][status][plugin:monitoring@5.0.0] Status changed from yellow to green - Ready log [19:46:06.971] [info][status][plugin:timelion@5.0.0] Status changed from uninitialized to green - Ready log [19:46:06.975] [info][listening] Server running at https://x.x.x.x:5601 log [19:46:06.975] [info][status][ui settings] Status changed from uninitialized to green - Ready

If I set logging.verbose: true in kibana.yml, when I send the search request, the new entries (notice the 502) are:

respons [19:45:13.751] POST /api/console/proxy?uri=_search 502 46ms - 9.0B log [19:45:13.784] [debug][plugin] Checking Elasticsearch version ops [19:45:13.967] memory: 84.5MB uptime: 0:01:21 load: [0.03 0.07 0.11] delay: 1.173 log [19:45:16.298] [debug][plugin] Checking Elasticsearch version respons [19:45:18.122] GET /api/reporting/jobs/list_completed_since?since=2016-11-02T19%3A37%3A34.544Z 200 10ms - 9.0B

Elasticsearch and Kibana and running on a separate Linux machine, through which I connect with the browser, all firewall/ports rules are open, without x-pack and ssl setup the search was working OK.

How can I resolve this problem?

2

If I just type https://x.x.x.x:9200/_search in my browser, I can accept the self-signed certifiate, provide the kibana username and password, and the result displays OK.

I can also send a curl search from the commandline (IP and password replaced with x here), it works also fine, so it is some kind of Kibana and X-pack config problem:

curl --user kibana:xxxx -k 'https://x.x.x.x:9200/_search'

results in (just like using the browser as above):

{"took":6,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":1,"max_score":1.0,"hits":[{"_index":".kibana","_type":"config","_id":"5.0.0","_score":1.0,"_source":{"buildNum":14438}}]}}

3

Hey,

have you configured the ca parameter, when using self signed certificates, in kibana? See the Enabling SSL docs..

--Alex

4

Update:

Yes, I provided paths to either the ca.crt or ca.key file (created by the certgen tool) for the elasticsearch.ssl.ca setting in kibana.yml.

I also set elasticsearch.ssl.verify: to true or false in kibana.yml.

I tried all possible combinations, everytime restarting Kibana, but the same problem remains in the "Dev Tools": Client request error: unable to verify the first certificate

Somebody else on github has opened an issue about this problem, see https://github.com/elastic/kibana/issues/8932. This looks strongly like a bug.

5

Not sure if this got resolved?

I had the same problem as well, racked my brain and trolled this forum and others for a solution. After bringing up my dilemma to my Sr. Sys Admin he pointed out where I went wrong. The solution is definitely certificate related. The directions given by Elastic on how one creates a CSR are spot on, however; how to apply the CRT was not so clear, at least to me.

The fix for me was to ensure I used the signed CSR certificate—this certificate is the CSR and CA that signed it. This is not the server private key cert.

I am using the rpm elastic stack build on a single server in dev, so my elastic config is along /etc path:
My elasticsearch.yml
xpack.ssl.key: /etc/elasticsearch/x-pack/privatekey.pem
xpack.ssl.key.passphrase: password
xpack.ssl.certifcate: /etc/elasticsearch/x-pack/publickey.pem
xpack.ssl.certifcate_authorities: /etc/elasticsearch/x-pack/signedCSR.pem
xpack.security.transport.ssl.enabled: true
xpack.security.http.ssl.enabled: true

My kibana.yml
server.ssl.cert: /etc/elasticsearch/x-pack/publickey.pem
server.ssl.key: /etc/elastcksearch/x-pack/privatekey.pem
elasticsearch.ssl.cert: /etc/elasticsearch/x-pack/publickey.pem
elasticsearch.ssl.key: /etc/elasticsearch/x-pack/privatekey.pem
elasticsearch.ssl.ca: /etc/elasticsearch/x-pack/signedCSR.pem

Hope this may help others troubleshoot their scenario, modify as needed

6

Thanks for the detailed reply!

Also it looks like the related fix https://github.com/elastic/kibana/pull/9823 as I understand should make it into 5.3.