Load Balancing Kibana

Elastic Stack Kibana
1

I 'm trying to load balance 3 kibana nodes running on aws behind a load balancer.

The load balancer needs a health check script in order to recognize the host as 'alive'. It then performs a round robin on the nodes that register as alive.

I tried putting a health check script into this directory:

/opt/kibana-4.2.1-linux-x64/src/ui/public

It's not even a script really. It's just a file. All the load balancer expects to do is perform an HTTP GET on this file and receive the word 'good' in response.

cat /opt/kibana/src/ui/public/healthcheck.php
good

I realize that kibana is written in nodejs!! But I'm lazy and I'd really rather not write another probe in the load balancer that will use a different file name. And it just so happens the majority of apps on this LB are PHP apps. The file could be named anything at all. Calling it foo.txt would work just as well!

But the LB is not able to find the health check even tho that file is in that directory.

I'm doing a proxy-pass in nginx in order to serve kibana:

server {
    listen 80;

    server_name logs1.example.com;


    location / {
        proxy_pass http://localhost:5601;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }

}

How can I get kibana to register the health check file with the LB?

Alternatively @spalger from IRC told me that I could also write a probe in the LB that would do a HEAD on /app/kibana on my kibana servers.

However when I attempt to do that, I'm getting 400 Bad Request error in response. Even tho Kibana seems to be running absolutely fine in a web browser. And I don't see any problem at all in the kibana logs:

HEAD http://logs1.example.com:5601/app/kibana
400 Bad Request
Cache-Control: no-cache
Connection: keep-alive
Date: Mon, 07 Mar 2016 20:31:38 GMT
Content-Type: application/json; charset=utf-8
Client-Date: Mon, 07 Mar 2016 20:31:38 GMT
Client-Peer: 52.72.210.180:5601
Client-Response-Num: 1
Kbn-Name: kibana
Kbn-Version: 4.4.1

I don't particularly care how I get this done. But I really would like to find a way to get Kibana4 load balanced. Any suggestions or tips on how to do that would be welcomed!

Thanks

2

@bluethundr I took another look at this and realized what the issue was: Kibana requires the kbn-version header be set for all non-GET requests...

We will fix this (issue is filed) but in the meantime if you can add headers to the request then simply ship the kibana version and this should start to work.

3

@spalger ok! I'll see if I can do that. If I have any questions on how to do this, I'll let you know!