Issues connecting kibana to elasticsearch cluster in kubernetes

Elastic Stack Kibana
1

I have an elasticsearch cluster on kubernetes and I'm trying to connect kibana to it. Currently this is the kibana.yml file that I mount into the config directory for my kibana pod

    server.name: kibana-logging
    server.host: 0.0.0.0
    elasticsearch.url: http://elasticsearch:9200
    elasticsearch.username: elastic
    elasticsearch.password: changeme
    xpack.monitoring.ui.container.elasticsearch.enabled: true

With this set up minikube service kibana, shows an ERR_TOO_MANY_REDIRECTS page.
Are there any changes required for my config? Thanks!

Investigating the logs for the kibana pod shows some very big requests with lots of repeating text (see followup reply)

2 
{"type":"log","@timestamp":"2017-10-12T18:48:42Z","tags":["listening","info"],"pid":1,"message":"Server running at http://0.0.0.0:5601"}
{"type":"log","@timestamp":"2017-10-12T18:48:42Z","tags":["status","ui settings","error"],"pid":1,"state":"red","message":"Status changed from uninitialized to red - Elasticsearch plugin is red","prevState":"uninitialized","prevMsg":"uninitialized"}
{"type":"log","@timestamp":"2017-10-12T18:48:45Z","tags":["status","plugin:ml@5.6.3","error"],"pid":1,"state":"red","message":"Status changed from yellow to red - Request Timeout after 3000ms","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
{"type":"response","@timestamp":"2017-10-12T18:51:30Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"192.168.99.100:32708","connection":"keep-alive","cache-control":"max-age=0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36","upgrade-insecure-requests":"1","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.8"},"remoteAddress":"172.17.0.1","userAgent":"172.17.0.1"},"res":{"statusCode":302,"responseTime":43,"contentLength":9},"message":"GET / 302 43ms - 9.0B"}
{"type":"response","@timestamp":"2017-10-12T18:51:30Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/api/v1/proxy/namespaces/default/services/kibana/login?next=%2Fapi%2Fv1%2Fproxy%2Fnamespaces%2Fdefault%2Fservices%2Fkibana%2F","method":"get","headers":{"host":"192.168.99.100:32708","connection":"keep-alive","cache-control":"max-age=0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36","upgrade-insecure-requests":"1","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.8"},"remoteAddress":"172.17.0.1","userAgent":"172.17.0.1"},"res":{"statusCode":302,"responseTime":10,"contentLength":9},"message":"GET /api/v1/proxy/namespaces/default/services/kibana/login?next=%2Fapi%2Fv1%2Fproxy%2Fnamespaces%2Fdefault%2Fservices%2Fkibana%2F 302 10ms - 9.0B"}
{"type":"response","@timestamp":"2017-10-12T18:51:30Z","tags":[],"pid":1,"method":"get","statusCode":302,"req":{"url":"/api/v1/proxy/namespaces/default/services/kibana/login?next=%2Fapi%2Fv1%2Fproxy%2Fnamespaces%2Fdefault%2Fservices%2Fkibana%2Fapi%2Fv1%2Fproxy%2Fnamespaces%2Fdefault%2Fservices%2Fkibana%2Flogin%3Fnext%3D%252Fapi%252Fv1%252Fproxy%252Fnamespaces%252Fdefault%252Fservices%252Fkibana%252F","method":"get","headers":{"host":"192.168.99.100:32708","connection":"keep-alive","cache-control":"max-age=0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36","upgrade-insecure-requests":"1","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.8"},"remoteAddress":"172.17.0.1","userAgent":"172.17.0.1"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET /api/v1/proxy/namespaces/default/services/kibana/login?next=%2Fapi%2Fv1%2Fproxy%2Fnamespaces%2Fdefault%2Fservices%2Fkibana%2Fapi%2Fv1%2Fproxy%2Fnamespaces%2Fdefault%2Fservices%2Fkibana%2Flogin%3Fnext%3D%252Fapi%252Fv1%252Fproxy%252Fnamespaces%252Fdefault%252Fservices%252Fkibana%252F 302 6ms - 9.0B"}

And the responses keep getting bigger in size as a continue scrolling through the logs

3

Curling the kibana service using curl -v results in the following response:

*   Trying 192.168.99.100...
* Connected to 192.168.99.100 (192.168.99.100) port 32708 (#0)
> GET / HTTP/1.1
> Host: 192.168.99.100:32708
> User-Agent: curl/7.43.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< location: /api/v1/proxy/namespaces/default/services/kibana/login?next=%2Fapi%2Fv1%2Fproxy%2Fnamespaces%2Fdefault%2Fservices%2Fkibana%2F
< kbn-name: kibana
< kbn-version: 5.6.3
< cache-control: no-cache
< content-length: 0
< Date: Thu, 12 Oct 2017 20:20:14 GMT
< Connection: keep-alive
< 
* Connection #0 to host 192.168.99.100 left intact

UPDATE:

I managed to get rid of the huge request test blocks but currently my requests to access the kibana service still hang. For now I disabled the login by setting xpack.security.enabled to false on both elasticsearch.yml and kibana.yml. But trying to curl the service or access it from chrome just hangs and now I don't see the requests being sent from my kibana pod as well

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.