Kibana server is not ready

[abhishek_Shetty]

Hi all,

I have installed latest version of Elastic search and Kibana on red hat 7.6, both services are running but I am unable to connect to kibana from browser which is giving Kibana server is not ready yet. I saw error and checked instances but no instance was found.

Can someone help in fixing the issue
Service Status:
sudo systemctl status kibana.service
● kibana.service - Kibana
Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2019-05-20 04:26:48 PDT; 4s ago
Main PID: 18508 (node)
Tasks: 11
CGroup: /system.slice/kibana.service
└─18508 /usr/share/kibana/bin/../node/bin/node --no-warnings --max-http-header-size=65536 /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml

May 20 04:26:48 scdl-sduipa-001 systemd[1]: Started Kibana.

ERROR LOG:
cat kibana.stdout
{"type":"log","@timestamp":"2019-05-20T05:57:13Z","tags":["fatal","root"],"pid":681,"message":"Error: Port 5601 is already in use. Another instance of Kibana may be running!\n at Root.shutdown (/usr/share/kibana/src/core/server/root/index.js:53:26)\n at Root.start (/usr/share/kibana/src/core/server/root/index.js:45:24)\n at process._tickCallback (internal/process/next_tick.js:68:7)\n at Function.Module.runMain (internal/modules/cjs/loader.js:745:11)\n at startup (internal/bootstrap/node.js:283:19)\n at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)"}

ps -ef | grep kibana
kibana 13724 1 99 03:52 ? 00:00:06 /usr/share/kibana/bin/../node/bin/node --no-warnings --max-http-header-size=65536 /usr/share/kibana/bin/../src/cli -c /etc/kibana/kibana.yml
root 13737 7105 0 03:52 pts/0 00:00:00 grep --color=auto kibana

curl -X GET "localhost:9200"
{
"name" : "scdl-sduipa-001",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "JkwpFVIkSs2-bo8vXey_jw",
"version" : {
"number" : "7.0.1",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "e4efcb5",
"build_date" : "2019-04-29T12:56:03.145736Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.7.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"

curl -X GET "localhost:9200/_cat/indices/.kib*?v&s=index"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

curl -X GET "localhost:5601"
curl: (7) Failed connect to localhost:5601; Connection refused

[tiagocosta]

@abhishek_Shetty according to logs, another process is already running into the port 5601.

Try to find what is the process running on that port with

lsof -i :5601

Then kill it and restart the kibana service. It should work then

[abhishek_Shetty]

Hi Tiagocosta,

I did checked the port and found one kibana process is running but frequently its PDI is changing and I am unable to kill the process. Is their any command or method we can fix this issue.

root@scdl-sduipa-001:/var/log/kibana# cat kibana.stderr

FATAL Error: Port 5601 is already in use. Another instance of Kibana may be running!

root@scdl-sduipa-001:/var/log/kibana# cat kibana.stdout
{"type":"log","@timestamp":"2019-05-20T05:57:13Z","tags":["fatal","root"],"pid":681,"message":"Error: Port 5601 is already in use. Another instance of Kibana may be running!\n at Root.shutdown (/usr/share/kibana/src/core/server/root/index.js:53:26)\n at Root.start (/usr/share/kibana/src/core/server/root/index.js:45:24)\n at process._tickCallback (internal/process/next_tick.js:68:7)\n at Function.Module.runMain (internal/modules/cjs/loader.js:745:11)\n at startup (internal/bootstrap/node.js:283:19)\n at bootstrapNodeJSCore (internal/bootstrap/node.js:743:3)"}
root@scdl-sduipa-001:/var/log/kibana# lsof -i :5601
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
node 6253 kibana 18u IPv4 1939985 0t0 TCP scdl-sduipa-001.fhlbsf-i.com:esmagent (LISTEN)

root@scdl-sduipa-001:/var/log/kibana# lsof -i :5601
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
node 6281 kibana 18u IPv4 1941176 0t0 TCP scdl-sduipa-001.fhlbsf-i.com:esmagent (LISTEN)

root@scdl-sduipa-001:/var/log/kibana# lsof -i :5601
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
node 6494 kibana 18u IPv4 1943812 0t0 TCP scdl-sduipa-001.fhlbsf-i.com:esmagent (LISTEN)

root@scdl-sduipa-001:/var/log/kibana# kill -9 6494
bash: kill: (6494) - No such process

[pathri (Vinit Pathri)]

i hope you are running kibana not as a service

Try killing kibana PID with below command and restart it

ps -ef | grep '.*node/bin/node.*src/cli'

Output:
root 6297 1 1 May18 ? 00:18:50 ../bin/../node/bin/node --no-warnings --max-http-header-size=65536 ../bin/../src/cli

kill -9 6297