I do not understand why 'kibana' does not work

ksb304 · November 2, 2018, 5:14am

Kibana does not work after rebooting the server.
I do not understand why 'kibana' does not work.
What's the problem?

warkolm · November 2, 2018, 5:18am

Please don't post pictures of text, they are difficult to read and some people may not be even able to see them
Are you able to repost just the text?

ksb304 · November 2, 2018, 5:30am

error [05:26:51.678] [warning][stats-collection] [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/_search","query":{"ignore_unavailable":true,"filter_path":"aggregations.types.buckets"},"body":"{"size":0,"query":{"terms":{"type":["dashboard","visualization","search","index-pattern","graph-workspace","timelion-sheet"]}},"aggs":{"types":{"terms":{"field":"type","size":6}}}}","statusCode":503,"response":"{"error":{"root_cause":,"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":},"status":503}"}
at respond (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\transport.js:307:15)
at checkRespForFailure (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\transport.js:266:7)
at HttpConnector. (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\connectors\http.js:159:7)
at IncomingMessage.bound (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\node_modules\lodash\dist\lodash.js:729:21)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickDomainCallback (internal/process/next_tick.js:218:9)
log [05:26:51.678] [warning][stats-collection] Unable to fetch data from kibana collector

The ES-server appears to be operating normally.

ksb304 · November 2, 2018, 5:44am

add

error [05:27:41.713] [warning][stats-collection] [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/_search","query":{"ignore_unavailable":true,"filter_path":"aggregations.types.buckets"},"body":"{"size":0,"query":{"terms":{"type":["dashboard","visualization","search","index-pattern","graph-workspace","timelion-sheet"]}},"aggs":{"types":{"terms":{"field":"type","size":6}}}}","statusCode":503,"response":"{"error":{"root_cause":,"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":},"status":503}"}
at respond (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\transport.js:307:15)
at checkRespForFailure (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\transport.js:266:7)
at HttpConnector. (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\connectors\http.js:159:7)
at IncomingMessage.bound (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\node_modules\lodash\dist\lodash.js:729:21)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickDomainCallback (internal/process/next_tick.js:218:9)

ksb304 · November 2, 2018, 6:04am

error [05:26:51.678] [warning][stats-collection] [search_phase_execution_exception] all shards failed :: {"path":"/.kibana/_search","query":{"ignore_unavailable":true,"filter_path":"aggregations.types.buckets"},"body":"{"size":0,"query":{"terms":{"type":["dashboard","visualization","search","index-pattern","graph-workspace","timelion-sheet"]}},"aggs":{"types":{"terms":{"field":"type","size":6}}}}","statusCode":503,"response":"{"error":{"root_cause":,"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":},"status":503}"}
at respond (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\transport.js:307:15)
at checkRespForFailure (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\transport.js:266:7)
at HttpConnector. (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\src\lib\connectors\http.js:159:7)
at IncomingMessage.bound (C:\Users\Administrator\Desktop\kibana1\node_modules\elasticsearch\node_modules\lodash\dist\lodash.js:729:21)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickDomainCallback (internal/process/next_tick.js:218:9)
log [05:26:51.678] [warning][stats-collection] Unable to fetch data from kibana collector

warkolm · November 2, 2018, 6:06am

Great, thanks.

Can you check your Elasticsearch logs as well? There should be something happening at the same time as these errors.

ksb304 · November 2, 2018, 6:29am

I get a 'time out' error during 'indexing'.

[2018-11-02T14:21:28,761][INFO ][o.e.n.Node [2018-11-02T14:21:28,917][INFO ][o.e.e.NodeEnvironment [2018-11-02T14:21:28,917][INFO ][o.e.e.NodeEnvironment [2018-11-02T14:21:57,065][INFO ][o.e.n.Node [2018-11-02T14:21:57,065][INFO ][o.e.n.Node [2018-11-02T14:21:57,065][INFO ][o.e.n.Node [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService [2018-11-02T14:22:03,940][INFO ][o.e.p.PluginsService [2018-11-02T14:22:08,925][INFO ][o.e.x.s.a.s.FileRolesStore] [2018-11-02T14:22:11,737][INFO ][o.e.x.m.j.p.l.CppLog [2018-11-02T14:22:11,847][DEBUG][o.e.a.ActionModule [2018-11-02T14:22:19,206][INFO ][o.e.d.DiscoveryModule [2018-11-02T14:22:20,175][INFO ][o.e.n.Node [2018-11-02T14:22:20,190][INFO ][o.e.n.Node [2018-11-02T14:22:20,394][INFO ][o.e.t.TransportService [2018-11-02T14:22:22,737][INFO ][o.e.b.BootstrapChecks [2018-11-02T14:22:25,922][INFO ][o.e.c.s.MasterService [2018-11-02T14:22:25,938][INFO ][o.e.c.s.ClusterApplierService] [2018-11-02T14:22:26,375][INFO ][o.e.x.s.t.n.Security [2018-11-02T14:22:26,375][INFO ][o.e.n.Node [2018-11-02T14:22:32,141][INFO ][o.e.c.r.a.AllocationService] [2018-11-02T14:22:58,342][WARN ][o.e.x.s.a.s.m.Native ] [cloudoc1] initializing ...
] [cloudoc1] using [1] data paths, mounts [[(C:)]], net usable_space [39.2gb], net total_space [111.3gb], types [NTFS]
] [cloudoc1] heap size [1007.3mb], compressed ordinary object pointers [true]
] [cloudoc1] node name [cloudoc1], node ID [iTnJtsgTRkaG9FUWpOrPkQ]
] [cloudoc1] version[6.4.0], pid[7164], build[default/zip/595516e/2018-08-17T23:18:47.308994Z], OS[Windows NT (unknown)/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14]
] [cloudoc1] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\2\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Delasticsearch, -Des.path.home=C:\Users\Administrator\Desktop\cloudoc1, -Des.path.conf=C:\Users\Administrator\Desktop\cloudoc1\config, -Des.distribution.flavor=default, -Des.distribution.type=zip]
] [cloudoc1] loaded module [aggs-matrix-stats]
] [cloudoc1] loaded module [analysis-common]
] [cloudoc1] loaded module [ingest-common]
] [cloudoc1] loaded module [lang-expression]
] [cloudoc1] loaded module [lang-mustache]
] [cloudoc1] loaded module [lang-painless]
] [cloudoc1] loaded module [mapper-extras]
] [cloudoc1] loaded module [parent-join]
] [cloudoc1] loaded module [percolator]
] [cloudoc1] loaded module [rank-eval]
] [cloudoc1] loaded module [reindex]
] [cloudoc1] loaded module [repository-url]
] [cloudoc1] loaded module [transport-netty4]
] [cloudoc1] loaded module [tribe]
] [cloudoc1] loaded module [x-pack-core]
] [cloudoc1] loaded module [x-pack-deprecation]
] [cloudoc1] loaded module [x-pack-graph]
] [cloudoc1] loaded module [x-pack-logstash]
] [cloudoc1] loaded module [x-pack-ml]
] [cloudoc1] loaded module [x-pack-monitoring]
] [cloudoc1] loaded module [x-pack-rollup]
] [cloudoc1] loaded module [x-pack-security]
] [cloudoc1] loaded module [x-pack-sql]
] [cloudoc1] loaded module [x-pack-upgrade]
] [cloudoc1] loaded module [x-pack-watcher]
] [cloudoc1] no plugins loaded
[cloudoc1] parsed [0] roles from file [C:\Users\Administrator\Desktop\cloudoc1\config\roles.yml]
MessageHandler] [controller/5948] [Main.cc@109] controller (64 bit): Version 6.4.0 (Build cf8246175efff5) Copyright (c) 2018 Elasticsearch BV
] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
] [cloudoc1] using discovery type [zen]
] [cloudoc1] initialized
] [cloudoc1] starting ...
] [cloudoc1] publish_address {192.168.1.99:9300}, bound_addresses {192.168.1.99:9300}
] [cloudoc1] bound or publishing to a non-loopback address, enforcing bootstrap checks
] [cloudoc1] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {cloudoc1}{iTnJtsgTRkaG9FUWpOrPkQ}{rbvASZJFSQGSobzqqfAhFQ}{192.168.1.99}{192.168.1.99:9300}{ml.machine_memory=8451579904, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[cloudoc1] new_master {cloudoc1}{iTnJtsgTRkaG9FUWpOrPkQ}{rbvASZJFSQGSobzqqfAhFQ}{192.168.1.99}{192.168.1.99:9300}{ml.machine_memory=8451579904, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {cloudoc1}{iTnJtsgTRkaG9FUWpOrPkQ}{rbvASZJFSQGSobzqqfAhFQ}{192.168.1.99}{192.168.1.99:9300}{ml.machine_memory=8451579904, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
Netty4HttpServerTransport] [cloudoc1] publish_address {192.168.1.99:9200}, bound_addresses {192.168.1.99:9200}
] [cloudoc1] started
[cloudoc1] updating number_of_replicas to [0] for indices [.kibana]
RoleMappingStore] [cloudoc1] Failed to clear cache for realms []

ksb304 · November 2, 2018, 6:33am

I get a 'time out' error during 'indexing'.

[2018-11-02T14:21:28,761][INFO ][o.e.n.Node ] [cloudoc1] initializing ...
[2018-11-02T14:21:28,917][INFO ][o.e.e.NodeEnvironment ] [cloudoc1] using [1] data paths, mounts [[(C:)]], net usable_space [39.2gb], net total_space [111.3gb], types [NTFS]
[2018-11-02T14:21:28,917][INFO ][o.e.e.NodeEnvironment ] [cloudoc1] heap size [1007.3mb], compressed ordinary object pointers [true]
[2018-11-02T14:21:57,065][INFO ][o.e.n.Node ] [cloudoc1] node name [cloudoc1], node ID [iTnJtsgTRkaG9FUWpOrPkQ]
[2018-11-02T14:21:57,065][INFO ][o.e.n.Node ] [cloudoc1] version[6.4.0], pid[7164], build[default/zip/595516e/2018-08-17T23:18:47.308994Z], OS[Windows NT (unknown)/10.0/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_111/25.111-b14]
[2018-11-02T14:21:57,065][INFO ][o.e.n.Node ] [cloudoc1] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=C:\Users\ADMINI~1\AppData\Local\Temp\2\elasticsearch, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Delasticsearch, -Des.path.home=C:\Users\Administrator\Desktop\cloudoc1, -Des.path.conf=C:\Users\Administrator\Desktop\cloudoc1\config, -Des.distribution.flavor=default, -Des.distribution.type=zip]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [aggs-matrix-stats]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [analysis-common]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [ingest-common]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [lang-expression]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [lang-mustache]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [lang-painless]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [mapper-extras]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [parent-join]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [percolator]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [rank-eval]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [reindex]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [repository-url]
[2018-11-02T14:22:03,878][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [transport-netty4]
[2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [tribe]
[2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-core]
[2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-deprecation]
[2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-graph]
[2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-logstash]
[2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-ml]
[2018-11-02T14:22:03,893][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-monitoring]
[2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-rollup]
[2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-security]
[2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-sql]
[2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-upgrade]
[2018-11-02T14:22:03,909][INFO ][o.e.p.PluginsService ] [cloudoc1] loaded module [x-pack-watcher]
[2018-11-02T14:22:03,940][INFO ][o.e.p.PluginsService ] [cloudoc1] no plugins loaded
[2018-11-02T14:22:08,925][INFO ][o.e.x.s.a.s.FileRolesStore] [cloudoc1] parsed [0] roles from file [C:\Users\Administrator\Desktop\cloudoc1\config\roles.yml]
[2018-11-02T14:22:11,737][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/5948] [Main.cc@109] controller (64 bit): Version 6.4.0 (Build cf8246175efff5) Copyright (c) 2018 Elasticsearch BV
[2018-11-02T14:22:11,847][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-11-02T14:22:19,206][INFO ][o.e.d.DiscoveryModule ] [cloudoc1] using discovery type [zen]
[2018-11-02T14:22:20,175][INFO ][o.e.n.Node ] [cloudoc1] initialized
[2018-11-02T14:22:20,190][INFO ][o.e.n.Node ] [cloudoc1] starting ...
[2018-11-02T14:22:20,394][INFO ][o.e.t.TransportService ] [cloudoc1] publish_address {192.168.1.99:9300}, bound_addresses {192.168.1.99:9300}
[2018-11-02T14:22:22,737][INFO ][o.e.b.BootstrapChecks ] [cloudoc1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-11-02T14:22:25,922][INFO ][o.e.c.s.MasterService ] [cloudoc1] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {cloudoc1}{iTnJtsgTRkaG9FUWpOrPkQ}{rbvASZJFSQGSobzqqfAhFQ}{192.168.1.99}{192.168.1.99:9300}{ml.machine_memory=8451579904, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2018-11-02T14:22:25,938][INFO ][o.e.c.s.ClusterApplierService] [cloudoc1] new_master {cloudoc1}{iTnJtsgTRkaG9FUWpOrPkQ}{rbvASZJFSQGSobzqqfAhFQ}{192.168.1.99}{192.168.1.99:9300}{ml.machine_memory=8451579904, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {cloudoc1}{iTnJtsgTRkaG9FUWpOrPkQ}{rbvASZJFSQGSobzqqfAhFQ}{192.168.1.99}{192.168.1.99:9300}{ml.machine_memory=8451579904, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]])
[2018-11-02T14:22:26,375][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [cloudoc1] publish_address {192.168.1.99:9200}, bound_addresses {192.168.1.99:9200}
[2018-11-02T14:22:26,375][INFO ][o.e.n.Node ] [cloudoc1] started
[2018-11-02T14:22:32,141][INFO ][o.e.c.r.a.AllocationService] [cloudoc1] updating number_of_replicas to [0] for indices [.kibana]
[2018-11-02T14:22:58,342][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [cloudoc1] Failed to clear cache for realms

system · November 30, 2018, 6:33am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.