Facing some issue building kibana

[niraj_pandey]

Hi ,

I am getting the following error messages on Kibana log:

og [10:47:37.026] [warning][admin][elasticsearch] Unable to revive connection: http://localhost:9229/
log [10:47:37.026] [warning][admin][elasticsearch] No living connections
log [10:47:39.533] [warning][admin][elasticsearch] Unable to revive connection: http://localhost:9229/
log [10:47:39.533] [warning][admin][elasticsearch] No living connections
log [10:47:39.534] [warning][admin][elasticsearch] Unable to revive connection: http://localhost:9229/
log [10:47:39.534] [warning][admin][elasticsearch] No living connections
log [10:47:42.085] [warning][admin][elasticsearch] Unable to revive connection: http://localhost:9229/
log [10:47:42.085] [warning][admin][elasticsearch] No living connections
log [10:47:42.124] [warning][admin][elasticsearch] Unable to revive connection: http://localhost:9229/

And on Kibana Dashboard it says:

Status Breakdown
ID Status
ui settings Elasticsearch plugin is red
plugin:kibana@5.5.1 Ready
plugin:elasticsearch@5.5.1 Unable to connect to Elasticsearch at http://localhost:9229.
plugin:xpack_main@5.5.1 Unable to connect to Elasticsearch at http://localhost:9229.
plugin:monitoring@5.5.1 Ready
plugin:searchprofiler@5.5.1 Unable to connect to Elasticsearch at http://localhost:9229.
plugin:ml@5.5.1 No Living connections
plugin:tilemap@5.5.1 Unable to connect to Elasticsearch at http://localhost:9229.
plugin:watcher@5.5.1 Unable to connect to Elasticsearch at http://localhost:9229.
plugin:grokdebugger@5.5.1 Ready
plugin:console@5.5.1 Ready
plugin:metrics@5.5.1 Ready
plugin:timelion@5.5.1 Ready

[niraj_pandey]

I am using version 5.5.1 for both Elasticsearch and Kibana

[niraj_pandey]

[niraj_pandey]

Here is my yml file:

elasticsearch.yml

xpack.security.enabled: false
xpack.graph.enabled: false
xpack.watcher.enabled: false

kibana.yml

pack.security.enabled: false
xpack.graph.enabled: false
xpack.reporting.enabled: false

elasticsearch.url: "http://localhost:9229" # connects to the proxy in front of elasticsearch
elasticsearch.username: "elastic"
elasticsearch.password: "password-for-proxy-in-front-of-elasticsearch"
server.host: "localhost"

If I replace 9229 port with 9200 in kibana.yml. I new error appearing

Access Denied
You are not authorized to access Monitoring. To use Monitoring, you need the privileges granted by both the kibana_user and monitoring_user roles.
If you are attempting to access a dedicated monitoring cluster, this might be because you are logged in as a user that is not configured on the monitoring cluster.

[cjcenizal]

Hmm, OK! I think we can figure this out. First, it seems like when Kibana's server is trying to connect to Elasticsearch, the proxy is failing to forward from port 9229 to Elasticsearch's port behind the proxy (9200). Are you connecting to Kibana directly, i.e. on localhost:5601? If so, can you try putting Kibana behind the proxy as well, and changing the config in kibana.yml to elasticsearch.url: "http://localhost:9200"?

Thanks,
CJ

[niraj_pandey]

Many Thanks !! It works.

One more question for you

On monitoring tab on Kibana , Elasticsearch "Health" status is yellow . Any idea what's wrong here , ideally it should be green.

Kibana log says:

log [03:13:44.197] [info][status][plugin:xpack_main@5.5.1] Status changed fr om yellow to green - Ready
log [03:13:44.199] [info][status][plugin:searchprofiler@5.5.1] Status change d from yellow to green - Ready
log [03:13:44.200] [info][status][plugin:tilemap@5.5.1] Status changed from yellow to green - Ready
log [03:13:44.203] [info][status][plugin:watcher@5.5.1] Status changed from yellow to green - Ready
log [03:13:44.210] [info][status][plugin:timelion@5.5.1] Status changed from uninitialized to green - Ready
log [03:13:44.241] [info][license][xpack] Imported license information from Elasticsearch for [monitoring] cluster: mode: trial | status: active | expiry da te: 2017-09-17T15:05:17+05:30
log [03:13:44.242] [info][listening] Server running at http://localhost:5601
log [03:13:44.247] [info][status][ui settings] Status changed from uninitial ized to green - Ready
log [03:13:44.252] [info][status][plugin:monitoring@5.5.1] Status changed fr om green to yellow - Waiting for Monitoring Health Check
log [03:13:44.339] [info][status][plugin:monitoring@5.5.1] Status changed fr om yellow to green - Ready

Elastisearch Log says:

[2017-08-22T08:51:32,105][INFO ][o.e.c.r.a.DiskThresholdMonitor] [Wm-GSOZ] low d isk watermark [85%] exceeded on [Wm-GSOZ9QAi-DGOTTPy2aA][Wm-GSOZ][/servers/scrat ch03/niraj/ELK/elasticsearch-5.5.1/data/nodes/0] free: 1.4tb[14.3%], replicas wi ll not be assigned to this node
[2017-08-22T08:52:02,202][INFO ][o.e.c.r.a.DiskThresholdMonitor] [Wm-GSOZ] low d isk watermark [85%] exceeded on [Wm-GSOZ9QAi-DGOTTPy2aA][Wm-GSOZ][/servers/scrat ch03/niraj/ELK/elasticsearch-5.5.1/data/nodes/0] free: 1.4tb[14.3%], replicas wi ll not be assigned to this node
[2017-08-22T08:52:32,472][INFO ][o.e.c.r.a.DiskThresholdMonitor] [Wm-GSOZ] low d isk watermark [85%] exceeded on [Wm-GSOZ9QAi-DGOTTPy2aA][Wm-GSOZ][/servers/scrat ch03/niraj/ELK/elasticsearch-5.5.1/data/nodes/0] free: 1.4tb[14.3%], replicas wi ll not be assigned to this node
[2017-08-22T08:53:02,572][INFO ][o.e.c.r.a.DiskThresholdMonitor] [Wm-GSOZ] low d isk watermark [85%] exceeded on [Wm-GSOZ9QAi-DGOTTPy2aA][Wm-GSOZ][/servers/scrat ch03/niraj/ELK/elasticsearch-5.5.1/data/nodes/0] free: 1.4tb[14.3%], replicas wi ll not be assigned to this node

[niraj_pandey]

Got the answer for this. Please ignore.

[niraj_pandey]

Can you let me know the procudure to change the default threshold limit from 85% to 90%.

Thanks
Niraj

[cjcenizal]

From the docs, it looks like you can customize these settings with a query like this:

PUT _cluster/settings
{
  "persisten": {
    "cluster.routing.allocation.disk.watermark.low": "80%",
    "cluster.routing.allocation.disk.watermark.high": "50gb",
    "cluster.info.update.interval": "1m"
  }
}

See Cluster update settings docs for more information on "persistent" vs "transient" settings.

Thanks,
CJ