Kibana cannot connect to ES cluster - Unable to retrieve version information from Elasticsearch nodes

[dorinand]

Kibana and ES version: 7.13.2

I am running cluster of 5 nodes (3 ES data & master nodes and 2 coordinating nodes) and one Kibana. Whole cluster is running in docker swarm on the same network. My ES cluster works fine, I can connect to it with Cerebro or Kibana from localhost. When I tried connect to ES from Kibana which is deployed to swarm, it ends up with:

2021-07-08T10:39:16.963979391Z {"type":"log","@timestamp":"2021-07-08T11:39:16+01:00","tags":["info","savedobjects-service"],"pid":953,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
2021-07-08T10:39:17.031184806Z {"type":"log","@timestamp":"2021-07-08T11:39:17+01:00","tags":["error","savedobjects-service"],"pid":953,"message":"Unable to retrieve version information from Elasticsearch nodes."}

I set ELASTICSEARCH_URL variable in Kibana and I am able to reach ES cluster from Kibana container with curl as:

• coordinating nodes service name - OK
• container IP - OK
• network alias - OK
• container name - OK

When I reach Kibana from browser, I can see:

Kibana server is not ready yet

I can see no kibana indices in ES.

When I run Kibana with command as docker run ... I am able to connect to ES cluster.

Questions:

1. Why I cannot connect to ES with Kibana from swarm but from localhost I can?
2. What is savedobjects-service? I have experience with older versions of ELK stack and do not remember logs like this.

[dorinand]

It was my mistake. I set ELASTICSEARCH_URL but Elasticsearch use variable ELASTICSEARCH_HOSTS. I would expect that Kibana will log something more than just Unable to retrieve version information from Elasticsearch nodes..

[alexandrpaliy (Alex)]

Funny thing, I faced the same problem an hour ago and found this topic, and now, when I figured this issue out, I also noticed you did it too

In my case, though, issue was caused by wrong kibana user.
I'm new to ELK, and I was trying to use user from "kibana_admin" group in order to let kibana connect to elasticsearch (the correct way is to use "kibana_system" user).

@dorinand

I would expect that Kibana will log something more than just Unable to retrieve version information from Elasticsearch nodes. .

I can't agree more.

[huang_jzx]

can you help me? i have same question
this is my ' kinana.yml':

server.name: kibana
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: [ "http://localhost:9200" ]

[dorinand]

Hi, if you have the same issue, check if you have elasticsearch on your localhost. Did you check it?