Delay starting Kibana until Elasticsearch fully up

Elastic Stack Kibana
1

Hi there,

It appears that I have a elasticsearch starting too slow for kibana, and tuning variables in config/kibana.yml didn't work. Any help in how to convince Kibana to wait longer for Elasticsearch to startup?

See the /var/log/kibana/kibana/stdout below:

{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":50,"err":"Request error, retrying -- connect ECONNREFUSED","msg":"","time":"2016-02-24T11:25:57.794Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"Unable to revive connection: http://localhost:9200/","time":"2016-02-24T11:25:57.810Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"No living connections","time":"2016-02-24T11:25:57.810Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":30,"msg":"Unable to connect to elasticsearch at http://localhost:9200. Retrying in 2.5 seconds.","time":"2016-02-24T11:25:57.817Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"Unable to revive connection: http://localhost:9200/","time":"2016-02-24T11:26:00.320Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"No living connections","time":"2016-02-24T11:26:00.320Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":30,"msg":"Unable to connect to elasticsearch at http://localhost:9200. Retrying in 2.5 seconds.","time":"2016-02-24T11:26:00.320Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"Unable to revive connection: http://localhost:9200/","time":"2016-02-24T11:26:02.821Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"No living connections","time":"2016-02-24T11:26:02.821Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":30,"msg":"Unable to connect to elasticsearch at http://localhost:9200. Retrying in 2.5 seconds.","time":"2016-02-24T11:26:02.821Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"Unable to revive connection: http://localhost:9200/","time":"2016-02-24T11:26:05.324Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"No living connections","time":"2016-02-24T11:26:05.325Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":30,"msg":"Unable to connect to elasticsearch at http://localhost:9200. Retrying in 2.5 seconds.","time":"2016-02-24T11:26:05.325Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"Unable to revive connection: http://localhost:9200/","time":"2016-02-24T11:26:07.926Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":40,"msg":"No living connections","time":"2016-02-24T11:26:07.926Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":30,"msg":"Unable to connect to elasticsearch at http://localhost:9200. Retrying in 2.5 seconds.","time":"2016-02-24T11:26:07.926Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":30,"msg":"Elasticsearch is still initializing the kibana index... Trying again in 2.5 second.","time":"2016-02-24T11:26:10.508Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":50,"err":{"message":"Request Timeout after 30000ms","name":"Error","stack":"Error: Request Timeout after 30000ms\n at null. (/opt/kibana-4.1.2-linux-x64/src/node_modules/elasticsearch/src/lib/transport.js:282:15)\n at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)"},"msg":"","time":"2016-02-24T11:26:43.044Z","v":0}
{"name":"Kibana","hostname":"kibana-elastic-host","pid":336,"level":60,"err":{"message":"Request Timeout after 30000ms","name":"Error","stack":"Error: Request Timeout after 30000ms\n at null. (/opt/kibana-4.1.2-linux-x64/src/node_modules/elasticsearch/src/lib/transport.js:282:15)\n at Timer.listOnTimeout [as ontimeout] (timers.js:112:15)"},"msg":"","time":"2016-02-24T11:26:43.044Z","v":0}

2

Hi,

Is Kibana exiting after that error, or is it still trying?

What version of Kibana are you running?

On the latest 4.4.1 release, with Elasticsearch not running, Kibana keeps trying forever. I think older versions did this as well. I'm testing with the default kibana.yml which has all the timeouts commented-out so they're using the default values.

What in your situation is causing both Elasticsearch and Kibana to be started at the same time? Is it when you reboot your system? If so, what OS are you running on?

Regards,
Lee

3

-Yes, Kibana totally EXITS after that last line

  • F*^#$%*(^ systemd starts them (perhaps ES after kibana)

  • Debian 8.3

  • kibana-4.12-linux-x64 can't go higher 'cause ElasticSearch 1.7.3 'cause
    of Logstash on the remote that we can't upgrade yet :frowning:

4

You don't need to upgrade Logstash if you are using the HTTP protocol.
So if you aren't then change and upgrade everything else :smiley:

5

That is the problem. Logstash is not using http ; (
Thus, how do I convince Kibana 4.1.2 to WAIT LONGER??

6

The change for LS to use HTTP is simple, the change you want for KB is not. Not without an upgrade when it'll just auto-retry.

7

warkolm http://discuss.elastic.co/users/warkolm Mark Walkom
http://discuss.elastic.co/users/warkolm
February 25

You don't need to upgrade Logstash if you are using the HTTP protocol.
So if you aren't then change and upgrade everything else [image: :smiley:]

Hmmm... is Logstash 1.5.4 able to connect to Elasticsearch (recent that is)
using HTTP?
I've just found the reason (bitten by Logstash 2.1 yet again) why I'm not
on the latest logstash:
the plugins used are dog slow on 2.1 while they all fly on 1.5.4 (I recall
tracking it to a jRuby issue)

SO the consensus is there are no way to have kibana wait a tad longer for
Elasticsearch when they are started together at boot?

Visit Topic
http://discuss.elastic.co/t/delay-starting-kibana-until-elasticsearch-fully-up/42580/4
or reply to this email to respond

Previous Replies
hvisage http://discuss.elastic.co/users/hvisage
February 25

-Yes, Kibana totally EXITS after that last line

F*^#$%*(^ systemd starts them (perhaps ES after kibana)

Debian 8.3

kibana-4.12-linux-x64 can't go higher 'cause Elasticsearch 1.7.3
'cause of Logstash on the remote that we can't upgrade yet [image:
:frowning:]

Visit Topic
http://discuss.elastic.co/t/delay-starting-kibana-until-elasticsearch-fully-up/42580/4
or reply to this email to respond

To stop receiving notifications for this particular topic, click here
http://discuss.elastic.co/t/delay-starting-kibana-until-elasticsearch-fully-up/42580/unsubscribe.
To unsubscribe from these emails, change your user preferences
http://discuss.elastic.co/my/preferences

Hendrik Visage
Envisage Systems / Envisage Cloud Solutions
084-612-5345 or 083-675-5506
hvisage@envisage.co.za

8

Not unless you hack it.