We are trying to learn how to setup security. We have TLS for the transport using certutil, we have TSL for http using a commercial cert, setup default account passwords and connected via Kibana using the elastic account. What we have tried in Kibana works except for Monitoring, it gets a popup error box
Monitoring Request Error
No Living connections: Check the Elasticsearch Monitoring cluster network connection and refer to the Kibana logs for more information.
HTTP 503
This may be related from kibana log, but there are 1000's of lines:
{"type":"error","@timestamp":"2019-03-07T23:03:09Z","tags":["error","monitoring-ui"],"pid":205462,"level":"error","error":{"message":"No Living connections","name":"Error","stack":"Error: No Living connections\n at sendReqWithConnection (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:226:15)\n at next (/usr/share/kibana/node_modules/elasticsearch/src/lib/connection_pool.js:214:7)\n at process._tickCallback (internal/process/next_tick.js:61:11)"},"message":"No Living connections"}
{"type":"log","@timestamp":"2019-03-07T23:03:09Z","tags":["warning","elasticsearch","monitoring-ui"],"pid":205462,"message":"Unable t
o revive connection: https://localhost:9200/"}
{"type":"log","@timestamp":"2019-03-07T23:03:09Z","tags":["warning","elasticsearch","monitoring-ui"],"pid":205462,"message":"No livin
g connections"}
{"type":"error","@timestamp":"2019-03-07T23:03:09Z","tags":["error","monitoring-ui"],"pid":205462,"level":"error","error":{"message":
"No Living connections","name":"Error","stack":"Error: No Living connections\n at sendReqWithConnection (/usr/share/kibana/node_mo
dules/elasticsearch/src/lib/transport.js:226:15)\n at next (/usr/share/kibana/node_modules/elasticsearch/src/lib/connection_pool.j
s:214:7)\n at process._tickCallback (internal/process/next_tick.js:61:11)"},"message":"No Living connections"}
{"type":"response","@timestamp":"2019-03-07T23:03:09Z","tags":,"pid":205462,"method":"post","statusCode":503,"req":{"url":"/api/monitoring/v1/clusters","method":"post","headers":{"host":"met-elk-exp6a.missouri.edu:5601","connection":"keep-alive","content-length":"81","accept":"application/json, text/plain, /","origin":"http://met-elk-exp6a.missouri.edu:5601","kbn-version":"6.6.1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36","content-type":"application/json;charset=UTF-8","referer":"http://met-elk-exp6a.missouri.edu:5601/app/monitoring","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9"},"remoteAddress":"161.130.254.132","userAgent":"161.130.254.132","referer":"http://met-elk-exp6a.missouri.edu:5601/app/monitoring"},"res":{"statusCode":503,"responseTime":117,"contentLength":9},"message":"POST /api/monitoring/v1/clusters 503 117ms - 9.0B"}