Kibana is installed successfully but not working, How to run kibana without xpack

Elastic Stack Kibana
1

I Installed elasticsearch 7.3 into 3 GCP server all 3 are master and also data nodes this is working fine, But now I installed kibana 7.3 to one of the server its status is green but when I try to open it from browser then then this is what I am getting.

Kibana server is not ready yet

also I want to access the kibana using internal ip I turn off the external ip.

this is my kibana.yml file

server.port: 5601
server.host: internal-ip-node-3
server.name: "la-stage-kibana"
elasticsearch.hosts: ["http://internal-ip-node1:9200","http://internal-ip-node-2:9200","http://internal-ip-node-3:9200"]
kibana.index: ".kibana"
elasticsearch.username: "kibana"
elasticsearch.password: "kibana"
elasticsearch.ssl.verificationMode: none
logging.dest: /etc/kibana/kibana.log

also this is kibana log

t after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:logstash@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:beats_management@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:maps@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:index_management@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:index_lifecycle_management@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:rollup@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:remote_clusters@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:cross_cluster_replication@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:file_upload@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:snapshot_restore@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:reporting@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"log","@timestamp":"2020-02-24T11:06:24Z","tags":["status","plugin:security@7.3.2","error"],"pid":5378,"state":"red","message":"Status changed from green to red - Request Timeout after 30000ms","prevState":"green","prevMsg":"Ready"}
{"type":"error","@timestamp":"2020-02-24T11:06:27Z","tags":["warning","process"],"pid":5378,"level":"error","error":{"message":"Error: Request Timeout after 30000ms\n    at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:362:15\n    at Timeout.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:391:7)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)","name":"UnhandledPromiseRejectionWarning","stack":"UnhandledPromiseRejectionWarning: Error: Request Timeout after 30000ms\n    at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:362:15\n    at Timeout.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:391:7)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)\n    at emitWarning (internal/process/promises.js:81:15)\n    at emitPromiseRejectionWarnings (internal/process/promises.js:120:9)\n    at process._tickCallback (internal/process/next_tick.js:69:34)"},"message":"Error: Request Timeout after 30000ms\n    at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:362:15\n    at Timeout.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:391:7)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)"}
{"type":"error","@timestamp":"2020-02-24T11:06:27Z","tags":["warning","process"],"pid":5378,"level":"error","error":{"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 255)","name":"UnhandledPromiseRejectionWarning","stack":"Error: Request Timeout after 30000ms\n    at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:362:15\n    at Timeout.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:391:7)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)"},"message":"Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 255)"}
{"type":"log","@timestamp":"2020-02-24T11:06:27Z","tags":["fatal","root"],"pid":5378,"message":"{ Error: Request Timeout after 30000ms\n    at /usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:362:15\n    at Timeout.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:391:7)\n    at ontimeout (timers.js:436:11)\n    at tryOnTimeout (timers.js:300:5)\n    at listOnTimeout (timers.js:263:5)\n    at Timer.processTimers (timers.js:223:10)\n  status: undefined,\n  displayName: 'RequestTimeout',\n  message: 'Request Timeout after 30000ms',\n  body: undefined,\n  isBoom: true,\n  isServer: true,\n  data: null,\n  output:\n   { statusCode: 503,\n     payload:\n      { statusCode: 503,\n        error: 'Service Unavailable',\n        message: 'Request Timeout after 30000ms' },\n     headers: {} },\n  reformat: [Function],\n  [Symbol(SavedObjectsClientErrorCode)]: 'SavedObjectsClient/esUnavailable' }"}
{"type":"log","@timestamp":"2020-02-24T11:06:32Z","tags":["info","plugins-system"],"pid":5493,"message":"Setting up [1] plugins: [translations]"}
{"type":"log","@timestamp":"2020-02-24T11:06:32Z","tags":["info","plugins","translations"],"pid":5493,"message":"Setting up plugin"}
{"type":"log","@timestamp":"2020-02-24T11:06:32Z","tags":["info","plugins-system"],"pid":5493,"message":"Starting [1] plugins: [translations]"}
2

I deleted all the nodes i.e

/var/lib/elasticsearch/nodes

and restart all the nodes of elasticsearch and also restarted kibana, now I am getting this.

kibana.log

{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:xpack_main@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:graph@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:spaces@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:searchprofiler@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:ml@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:tilemap@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:watcher@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:grokdebugger@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:logstash@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:beats_management@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:maps@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:index_management@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:index_lifecycle_management@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:rollup@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:remote_clusters@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:cross_cluster_replication@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:file_upload@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:snapshot_restore@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:reporting@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:security@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:31Z","tags":["status","plugin:elasticsearch@7.3.2","error"],"pid":9014,"state":"red","message":"Status changed from yellow to red - Request Timeout after 30000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"log","@timestamp":"2020-02-24T11:45:33Z","tags":["warning","task_manager"],"pid":9014,"message":"PollError Request Timeout after 30000ms"}
{"type":"log","@timestamp":"2020-02-24T11:45:33Z","tags":["warning","maps"],"pid":9014,"message":"Error scheduling telemetry task, received NotInitialized: Tasks cannot be scheduled until after task manager is initialized!"}
{"type":"log","@timestamp":"2020-02-24T11:45:33Z","tags":["warning","telemetry"],"pid":9014,"message":"Error scheduling task, received NotInitialized: Tasks cannot be scheduled until after task manager is initialized!"}
{"type":"log","@timestamp":"2020-02-24T11:46:01Z","tags":["license","warning","xpack"],"pid":9014,"message":"License information from the X-Pack plugin could not be obtained from Elasticsearch for the [data] cluster. Error: Request Timeout after 30000ms"}
3

Your config looks fine to me, it looks like Kibana can't reach Elasticsearch.

Could you share the Elasticsearch config as well? By default Elasticsearch only allows access from localhost, in your case you might need _local_ or _site_ to allow access from the Kibana host.

Another solution and also a common setup is to run an Elasticsearch coordinating node on the same host as Kibana and let Kibana only connect to the local node - this gives Elasticsearch as much control as possible and should also work around this problem.

4

Thanks for the reply but it start working when I reload the page several time, and also I change elasticsearch.hosts in kibana.yml file:

elasticsearch.hosts: ["http://internal-ip-node1:9200"]

i.e: Ip of node in which kibana installed, I just removed the other nodes IP, I don't this is right changes that I should do or it just start working by keep reloading the page I don't know the exact reason.

5

Maybe there are some other routing issues in your GCP setup, but if it works like this I would keep it that way as it's the recommended setup anyway.

Kibana takes a while to start up, so a little while of Kibana server is not ready yet is expected - after this it should work consistently. If you are restarting your Elasticsearch nodes at the same time, Kibana will also wait with its start routine till Elasticsearch is available, so this could delay things further.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.