Kibana error "Kibana server is not ready yet"

Hello guys.

I got "Kibana server is not ready yet" message while setting ELK-stack. This is my second trial but got this again, and I even couldn’t get any clue on this yet.

Elasticsearch and kibana version is 7.3.2 and i'm running this on CentOs7.
curl localhost:5601 results in Kibana server is not ready yet.
So it's impossible to open kibana in web.

This is elasticsearch.yml setting.

[root@es-test-new ~]# cat /etc/elasticsearch/elasticsearch.yml
cluster.name: estest-new
node.name: mynode
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
http.cors.enabled: true
http.cors.allow-origin: "*"
discovery.seed_hosts: [ “x.x.x.x:9300", ] ######## x.x.x.x is just my IP.
cluster.initial_master_nodes: [ “x.x.x.x:9300", ]

This is the one of Kibana.

[root@es-test-new ~]# cat /etc/kibana/kibana.yml
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://x.x.x.x:9200"] ######## x.x.x.x is just my IP.

When I checked kibana.stderr, it said port 5601 is already in use (so I once set server.port to 5602 but it didn’t help.)

This is firewall rule in GCP.

And this is the result from netstat and lsof commend.
[root@es-test-new ~]# netstat -tnlp

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1475/master
tcp 0 0 0.0.0.0:5601 0.0.0.0:* LISTEN 32277/node
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1528/sshd
tcp6 0 0 ::1:25 :::* LISTEN 1475/master
tcp6 0 0 :::9200 :::* LISTEN 32151/java
tcp6 0 0 :::9300 :::* LISTEN 32151/java
tcp6 0 0 :::22 :::* LISTEN 1528/sshd`

[root@es-test-new kibana]# lsof -i -nP | grep LISTEN
java 1370 elasticsearch 214u IPv6 20600 0t0 TCP *:9300 (LISTEN)
java 1370 elasticsearch 240u IPv6 18091 0t0 TCP *:9200 (LISTEN)
master 1488 root 13u IPv4 17368 0t0 TCP 127.0.0.1:25 (LISTEN)
master 1488 root 14u IPv6 17369 0t0 TCP [::1]:25 (LISTEN)
sshd 1533 root 3u IPv4 17967 0t0 TCP *:22 (LISTEN)
sshd 1533 root 4u IPv6 17969 0t0 TCP *:22 (LISTEN)
node 2142 kibana 18u IPv4 23571 0t0 TCP *:5601 (LISTEN)`

Elasticsearch is running well as you can see below.

curl localhost:9200
{
"name" : "mynode",
"cluster_name" : "estest-new",
"cluster_uuid" : "na",
"version" : {
"number" : "7.3.2",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "1c1faf1",
"build_date" : "2019-09-06T14:40:30.409026Z",
"build_snapshot" : false,
"lucene_version" : "8.1.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}

And I also check the version of Kibana.
/usr/share/kibana/bin/kibana --version --allow-root
7.3.2

So this is not the problem of version.

When I check the status, I could find both ES and Kibana are running.
systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since 2019-11-03 22:30:05 KST; 12h ago
Docs: http://www.elastic.co
Main PID: 32151 (java)
CGroup: /system.slice/elasticsearch.service
├─32151 /usr/share/elasticsearch/jdk/bin/java -Xms2g -Xmx2g -XX:+U...
└─32244 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-...

systemctl status kibana
● kibana.service - Kibana
Loaded: loaded (/etc/systemd/system/kibana.service; enabled; vendor preset: disabled)
Active: active (running) since 2019-11-03 22:30:28 KST; 12h ago
Main PID: 32277 (node)
CGroup: /system.slice/kibana.service
└─32277 /usr/share/kibana/bin/../node/bin/node --no-warnings --max...

Can you guess why this is happening?
I would be so grateful if you give me some help.
Thanks in advance.

Hi @HyunLee, I'm not 100% sure, but from the error message and from the systemctl and lsof seems that you have two kibana instances running, one with PID 2142 and another with PID 32277.

Can you please try to kill the process 2142 manually and restart the kibana service?
It's possible that maybe you started both the service and manually an instance of kibana