Issues in startup kibana

Elastic Stack Kibana
1

we started configuring ES in production environment and i see below exception in the ES logs when i start kibana, I followed similar installation procedure like i did in lower environments not sure what the issue is

[2017-03-04T21:30:55,748][DEBUG][o.e.a.s.TransportSearchAction] [master01] All shards failed for phase: [query_fetch]
org.elasticsearch.action.NoShardAvailableActionException: null
at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:115) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:154) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:53) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:173) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:145) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:87) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:75) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:64) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:530) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.rest.action.search.RestSearchAction.lambda$prepareRequest$1(RestSearchAction.java:81) ~[elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:82) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:162) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.http.HttpServer.dispatchRequest(HttpServer.java:115) [elasticsearch-5.2.2.jar:5.2.2]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:515) [transport-netty4-5.2.2.jar:5.2.2]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:70) [transport-netty4-5.2.2.jar:5.2.2]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:65) [transport-netty4-5.2.2.jar:5.2.2]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]

here is my journal logs for kibana, i have disabled x-pack on both ES & Kibana

Mar 04 21:15:03 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:03Z","tags":["status","plugin:elasticsearch@5.2.2","error"],"pid":24369,"state":"red","message":"Status
Mar 04 21:15:03 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:03Z","tags":["status","plugin:tilemap@5.2.2","error"],"pid":24369,"state":"red","message":"Status change
Mar 04 21:15:03 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:03Z","tags":["status","plugin:searchprofiler@5.2.2","error"],"pid":24369,"state":"red","message":"Status
Mar 04 21:15:03 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:03Z","tags":["status","plugin:xpack_main@5.2.2","error"],"pid":24369,"state":"red","message":"Status cha
Mar 04 21:15:00 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:00Z","tags":["status","ui settings","error"],"pid":24369,"state":"red","message":"Status changed from ye
Mar 04 21:15:00 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:00Z","tags":["status","plugin:elasticsearch@5.2.2","error"],"pid":24369,"state":"red","message":"Status
Mar 04 21:15:00 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:00Z","tags":["status","plugin:tilemap@5.2.2","error"],"pid":24369,"state":"red","message":"Status change
Mar 04 21:15:00 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:00Z","tags":["status","plugin:searchprofiler@5.2.2","error"],"pid":24369,"state":"red","message":"Status
Mar 04 21:15:00 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:15:00Z","tags":["status","plugin:xpack_main@5.2.2","error"],"pid":24369,"state":"red","message":"Status cha
Mar 04 21:14:38 ipselsprd01 kibana[24369]: {"type":"response","@timestamp":"2017-03-05T02:14:38Z","tags":,"pid":24369,"method":"get","statusCode":404,"req":{"url":"/favicon.ico","method":"ge
Mar 04 21:14:38 ipselsprd01 kibana[24369]: {"type":"response","@timestamp":"2017-03-05T02:14:38Z","tags":,"pid":24369,"method":"get","statusCode":404,"req":{"url":"/favicon.ico","method":"ge
Mar 04 21:14:38 ipselsprd01 kibana[24369]: {"type":"response","@timestamp":"2017-03-05T02:14:37Z","tags":,"pid":24369,"method":"get","statusCode":500,"req":{"url":"/app/kibana","method":"get
Mar 04 21:14:38 ipselsprd01 kibana[24369]: de_modules/bluebird/js/main/promise_array.js:92:18)\n at ReductionPromiseArray.init (/usr/share/kibana/node_modules/bluebird/js/main/reduce.js:42:
Mar 04 21:14:38 ipselsprd01 kibana[24369]: {"type":"error","@timestamp":"2017-03-05T02:14:37Z","tags":,"pid":24369,"level":"error","message":"Cannot read property 'toJSON' of undefined","err
Mar 04 21:14:38 ipselsprd01 kibana[24369]: {"type":"log","@timestamp":"2017-03-05T02:14:38Z","tags":["warning","process"],"pid":24369,"level":"error","message":"Unhandled promise rejection (re

2

There should be more to that ES error.

3

Mark, i have updated with more, if i stop kibana then i don't see any of this in ES log, also i have x-pack monitoring attributes disabled in both ES and Kibana yml,

i see error just after below line

[2017-03-06T16:17:22,824][INFO ][o.e.c.m.MetaDataCreateIndexService] [master01] [ipselsprd01.kibana] creating index, cause [api], templates [], shards [1]/[1], mappings [server, config]

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.