Unable to revive connection: http://localhost:9200 with Docker

gcio · January 11, 2021, 1:19pm

Hi everyone, I'm starting a docker container as follows

docker run -d --rm --name ejq-elk -v $PWD:/var/backups -p 5601:5601 sebp/elk:7.10.0

However, this returns a connection error


==> /var/log/kibana/kibana5.log <==

{"type":"log","@timestamp":"2021-01-11T13:12:42Z","tags":["error","elasticsearch","data"],"pid":338,"message":"[ConnectionError]: connect ECONNREFUSED 127.0.0.1:9200"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","elasticsearch","monitoring"],"pid":338,"message":"Unable to revive connection: http://localhost:9200/"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","elasticsearch","monitoring"],"pid":338,"message":"No living connections"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","plugins","licensing"],"pid":338,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","elasticsearch","data"],"pid":338,"message":"Unable to revive connection: http://localhost:9200/"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","elasticsearch","data"],"pid":338,"message":"No living connections"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","plugins","licensing"],"pid":338,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","elasticsearch","data"],"pid":338,"message":"Unable to revive connection: http://localhost:9200/"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["warning","elasticsearch","data"],"pid":338,"message":"No living connections"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["error","plugins","taskManager","taskManager"],"pid":338,"message":"Failed to poll for work: Error: No Living connections"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["error","elasticsearch","data"],"pid":338,"message":"[ConnectionError]: connect ECONNREFUSED 127.0.0.1:9200"}

{"type":"log","@timestamp":"2021-01-11T13:12:44Z","tags":["error","elasticsearch","data"],"pid":338,"message":"[ConnectionError]: connect ECONNREFUSED 127.0.0.1:9200"}

Does anyone know why this might happen?

Thank you in advance

majagrubic · January 11, 2021, 1:31pm

Is your ES cluster up and running?

gcio · January 11, 2021, 1:37pm

Hi Maja, thank you for your answer.

I think so:

 * Starting periodic command scheduler cron
   ...done.
 * Starting Elasticsearch Server
   ...done.
waiting for Elasticsearch to be up (1/30)
waiting for Elasticsearch to be up (2/30)
waiting for Elasticsearch to be up (3/30)
waiting for Elasticsearch to be up (4/30)
waiting for Elasticsearch to be up (5/30)
waiting for Elasticsearch to be up (6/30)
waiting for Elasticsearch to be up (7/30)
waiting for Elasticsearch to be up (8/30)
Waiting for Elasticsearch cluster to respond (1/30)
logstash started.
 * Starting Kibana5
   ...done.

Is this what you mean?
G

gcio · January 11, 2021, 1:40pm

This is another part of the log, not sure if it helps

==> /var/log/kibana/kibana5.log <==
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: write EPIPE"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"Request error, retrying\nPUT http://localhost:9200/.apm-custom-link => socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"Request error, retrying\nPUT http://localhost:9200/.apm-agent-configuration => socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["warning","plugins","telemetry"],"pid":330,"message":"Unable to update legacy telemetry configs."}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["warning","plugins","apm"],"pid":330,"message":"Failed to fetch saved telemetry data."}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["warning","environment"],"pid":330,"message":"Detected an unhandled Promise rejection.\nConnectionError: socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"Request error, retrying\nGET http://localhost:9200/_ilm/policy/kibana-event-log-policy => socket hang up"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: connect ECONNREFUSED 127.0.0.1:9200"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: connect ECONNREFUSED 127.0.0.1:9200"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: connect ECONNREFUSED 127.0.0.1:9200"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"[ConnectionError]: connect ECONNREFUSED 127.0.0.1:9200"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["error","elasticsearch","data"],"pid":330,"message":"Request error, retrying\nPOST http://localhost:9200/.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true&max_docs=10&conflicts=proceed => connect ECONNREFUSED 127.0.0.1:9200"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["warning","elasticsearch","data"],"pid":330,"message":"Unable to revive connection: http://localhost:9200/"}
{"type":"log","@timestamp":"2021-01-11T13:36:39Z","tags":["warning","elasticsearch","data"],"pid":330,"message":"No living connections"}

majagrubic · January 11, 2021, 1:45pm

The error you're receiving is because Kibana cannot establish connection to Elasticsearch. Possible root cause is that something is wrong in Kibana configuration or that the Elasticsearch cluster didn't start properly. Could you check the health of ES cluster?
curl -X GET "localhost:9200/_cluster/health?pretty"

gcio · January 11, 2021, 1:48pm

Here you go. It seems that it cannot connect to 9200 at all.

(base) $:~ curl -X GET "localhost:9200/_cluster/health?pretty"
curl: (7) Failed to connect to localhost port 9200: Connection refused

majagrubic · January 11, 2021, 1:53pm

Just to be clear: are you running the Docker Elasticsearch image at all? You should have two Docker containers running in parallel: one for Elasticsearch and for Kibana. Or did you just run Kibana image without running Elasticsearch as well?

gcio · January 11, 2021, 2:01pm

Ehm, no

OK, trying now following this https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html

I'll let you know how it goes. Thank you so much.
G

Jurilz · January 11, 2021, 2:52pm

If your Kibana is running in a docker container, than for this Kibana instance "localhost" is not your machine (which you are forwarding the 9200 port to) but it's container, and in the Kibana-Docker-Container there is no Elasticsearch under localhost:9200.

In this case you have to put both containers in the same (docker)-network and address elasticsearch like this name-of-elasticsarch-container:9200 from the Kibana config.

gcio · January 11, 2021, 2:54pm

Thank you Maja and Jurilz, it works now!

All the best,
G

system · February 8, 2021, 2:54pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Kibana Error Connecting to ES on Docker - Cannot Revive Connection Kibana	3	16577	June 22, 2018
How to resolve Unable to revive connection and No living connections Kibana docker	2	6068	October 7, 2020
Kibana Unable to revive connection. No living connections Kibana	5	34915	February 26, 2019
HELP! Kibana Error Connecting to ES on Docker - Cannot Revive Connection Kibana	3	1712	September 6, 2019
Kibana "Unable to revive connection" to Elasticsearch on Docker Elasticsearch	4	4680	May 12, 2020

Unable to revive connection: http://localhost:9200 with Docker

Related topics