X-pack monitoring using Kibana error at start up => self signed certificate in certificate chain

[Tamilselvan]

Hi,

Here's what I am trying to do (setting up the xpack monitoring feature),

I am using elastic search 5.2.1 protected with x-pack 5.2.1.
My test set up has 2 data node, 1 master and 1 client nodes.
I am running my kibana(5.2.1) server in the same machine where I am running the client node.

I have set up a separate single node cluster in a different machine and was pointing my kibana monitoring to that url

xpack.monitoring.elasticsearch.url: "https://xxx.xx.xx.xx:9200"
xpack.monitoring.elasticsearch.username: "kibana-user"
xpack.monitoring.elasticsearch.password: "password"
When I login to the kibana UI I don't see the tabs at the top and I see the status as RED.

kobana log shows:

{"type":"log","@timestamp":"2017-02-28T19:10:37Z","tags":["plugins","debug"],"pid":25817,"plugin":{"name":"spy_modes","version":"kibana"},"message":"Initializing plugin spy_modes@kibana"}
{"type":"log","@timestamp":"2017-02-28T19:10:37Z","tags":["plugins","debug"],"pid":25817,"plugin":{"name":"status_page","version":"kibana"},"message":"Initializing plugin status_page@kibana"}
{"type":"log","@timestamp":"2017-02-28T19:10:37Z","tags":["plugins","debug"],"pid":25817,"plugin":{"name":"table_vis","version":"kibana"},"message":"Initializing plugin table_vis@kibana"}
{"type":"log","@timestamp":"2017-02-28T19:10:37Z","tags":["plugins","debug"],"pid":25817,"plugin":{"name":"tagcloud","version":"kibana"},"message":"Initializing plugin tagcloud@kibana"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["plugins","debug"],"pid":25817,"plugin":{"author":"Rashid Khan rashid@elastic.co","name":"timelion","version":"kibana"},"message":"Initializing plugin timelion@kibana"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["error","elasticsearch","monitoring-ui"],"pid":25817,"message":"Request error, retrying\nHEAD https://xxx.xx.xx.xxx:9200/ => self signed certificate in certificate chain"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["status","plugin:timelion@5.2.1","info"],"pid":25817,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["listening","info"],"pid":25817,"message":"Server running at https://xxx.xx.xx.xx:5601"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["status","ui settings","info"],"pid":25817,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["warning","elasticsearch","monitoring-ui"],"pid":25817,"message":"Unable to revive connection: https://xxx.xx.xx.xxx:9200/"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["warning","elasticsearch","monitoring-ui"],"pid":25817,"message":"No living connections"}
{"type":"log","@timestamp":"2017-02-28T19:10:39Z","tags":["status","plugin:monitoring@5.2.1","error"],"pid":25817,"state":"red","message":"Status changed from yellow to red - No Living connections","prevState":"yellow","prevMsg":"Waiting for Monitoring Health Check"}
{"type":"log","@timestamp":"2017-02-28T19:10:40Z","tags":["plugin","debug"],"pid":25817,"message":"Checking Elasticsearch version"}

[Brandon_Kobel]

@Tamilselvan if the certificate that you're using with Elasticsearch is a self-signed certificate, you'll have to set the xpack.monitoring.elasticsearch.ssl.ca: /path/to/cert.crt setting in the kibana.yml

[Tamilselvan]

@Brandon_Kobel,

Thanks a ton. You saved me a day!!

I was having the ca configuration as 'elasticsearch.ssl.ca: /path/to/cert.crt' without the xpack.monitoring prefix.
You hit right at the point.

Thanks!!!

[Tamilselvan]

Hi @Brandon_Kobel,

I am now receiving the below error while trying to query the

Client request error: Hostname/IP doesn't match certificate's altnames: "IP: xxx.xx.xxx.xx is not in the cert's list:
In the chrome developer console, I see Bad Gateway (502) error code.

From https://github.com/elastic/kibana/issues/8932 - it looks like a known issue.

I have the below config set up in my kibana.yml

console.enabled: true
console.proxyFilter: .*
console.proxyConfig: 
  - match: 
      protocol: https
      host: "*"
      port: "{9200..9299}"
      
    ssl: 
      ca: /etc/kibana/path/ca.pem
      verify: false
      
  - timeout: 180000

Is there a workaround? Is it fixed in the current version 5.2.1 / 5.2.2?

[Brandon_Kobel]

That issue that you linked to is for the 'Dev Tools' application inside of Kibana itself:

Where exactly are you seeing that error? Could you post a screenshot?

[Tamilselvan]

@Brandon_Kobel
Here's the screenshot

I have the console.proxyConfig.0.ssl.verify: false, which is expected to skip the certificate hostname validation, but that doesn't seem to be the case.

For now as a workaround, I have modified the console/index.js - default setting for verify to false, to make it working.

But would like to confirm whether the console setting being not picked is a bug and when (which version) would the fix be out?

[Brandon_Kobel]

@Tamilselvan version 5.3.0 will remove the need to specify the console.proxyConfig section and will instead use the values specified in Elasticsearch config section.

[Tamilselvan]

Yes, Thats what I read from one of the topics in the forum.
Thanks for the confirmation!!