# Kibana can't connect to ES having (openstack) LoadBalancer setting

**URL:** <https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691>\
**Category:** Elastic Cloud on Kubernetes (ECK)\
**Created:** [November 5, 2019, 11:20pm UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691 "2019-11-05T23:20:24Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![JuYeong\_Park](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juyeong_park/32/45092_2.png) [@JuYeong\_Park](https://discuss.elastic.co/u/JuYeong_Park)\
**Post date:** [November 5, 2019, 11:20pm UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691/1 "2019-11-05T23:20:24Z")

</div>

I want to use ES and Kibana on LoadBalancer Env. So, i added load balancer options to ES yaml as below.

```
apiVersion: elasticsearch.k8s.elastic.co/v1beta1
kind: Elasticsearch
metadata:
  name: lb
spec:
  http:
    service:
       metadata:
         annotations:
           service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
       spec:
          externalTrafficPolicy: Local
          type: LoadBalancer
...

```

And then, i executed Kibana yaml(default) as below.

```
apiVersion: kibana.k8s.elastic.co/v1beta1
kind: Kibana
metadata:
  name: test
spec:
  version: 6.8.0
  count: 1
  elasticsearchRef:
    name: "lb"
  podTemplate:
    spec:
      containers:
      - name: kibana
        resources:
          limits:
            memory: 1Gi
            cpu: 1

```

On this settings, kibana coudln't be executed normally having error as below.

kubernetes error  
`Warning Unhealthy 2m43s (x179 over 32m) kubelet, dkosv3-ingress-test-worker-4 Readiness probe failed: HTTP probe failed with statuscode: 503`

docker error  
`{"type":"log","@timestamp":"2019-11-05T22:57:32Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: https://lb-es-http.default.svc:9200/"} {"type":"log","@timestamp":"2019-11-05T22:57:32Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"No living connections"} {"type":"log","@timestamp":"2019-11-05T22:57:32Z","tags":["warning","task_manager"],"pid":1,"message":"PollError No Living connections"} `

So, i want to know that what options is needed to use kibana for es having load balancer settings.

---

<div class="post-metadata">

**Author:** ![sebgl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebgl/32/48702_2.png) [@sebgl](https://discuss.elastic.co/u/sebgl)\
**Post date:** [November 7, 2019, 2:35pm UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691/2 "2019-11-07T14:35:18Z")

</div>

Hey @JuYeong_Park,

A few questions to better understand the problem:

- Are you able to access the Elasticsearch API (without Kibana)?
- Are your Elasticsearch Pods up and running?
- Do things work without the LoadBalancer service?

---

<div class="post-metadata">

**Author:** ![JuYeong\_Park](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juyeong_park/32/45092_2.png) [@JuYeong\_Park](https://discuss.elastic.co/u/JuYeong_Park)\
**Post date:** [November 10, 2019, 10:26pm UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691/3 "2019-11-10T22:26:25Z")

</div>

@sebgl

Thank you for your response.

1. Are you able to access the Elasticsearch API (without Kibana)?  
Yes. I can access Elasticsearch on EXTERNAL-IP.

> curl -u "elastic:$PASSWORD" -k "http://EXTERNAL\_IP:9200"

responds right result.

1. Are your Elasticsearch Pods up and running?  
Yes. All pods and service are running status.

2. Do things work without the LoadBalancer service?  
Yes. When i used Elasticsearch without LB service, kibana could access Elasticsearch normally. But, when i use Elasticsearch with LB, then kibana pod has a status like below image.

![image](https://us1.discourse-cdn.com/elastic/original/3X/b/4/b40c67d8d68f45eaaa3bfa3bfdb5a58ae2be87d8.png)

Below error saying, i think that Kibana can't find accurate host of "lb-es-http.default.svc". Maybe mapping error?

> {"type":"log","@timestamp":"2019-11-05T22:57:32Z","tags":["warning","elasticsearch","admin"],"pid":1,"message":"Unable to revive connection: [https://lb-es-http.default.svc:9200/](https://lb-es-http.default.svc:9200/)"}

---

<div class="post-metadata">

**Author:** ![sebgl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sebgl/32/48702_2.png) [@sebgl](https://discuss.elastic.co/u/sebgl)\
**Post date:** [November 18, 2019, 11:44am UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691/4 "2019-11-18T11:44:43Z")

</div>

Interesting. According to your experience it looks like Openstack internal LoadBalancer prevents the internal service DNS from resolving. Which would make sense, I guess, if you want another "internal" endpoint.

If that helps, you should be able to keep the existing Elasticsearch service with default options, so Kibana can reach it normally. Then create your own additional Service to target Elasticsearch Pods the way you want:

```auto
apiVersion: v1
kind: Service
metadata:
  name: your-service-name-here
  annotations:
    service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  ports:
  - name: https
    port: 9200
    protocol: TCP
    targetPort: 9200
  selector:
    common.k8s.elastic.co/type: elasticsearch
    elasticsearch.k8s.elastic.co/cluster-name: elasticsearch-sample

```

---

<div class="post-metadata">

**Author:** ![JuYeong\_Park](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juyeong_park/32/45092_2.png) [@JuYeong\_Park](https://discuss.elastic.co/u/JuYeong_Park)\
**Post date:** [November 19, 2019, 5:00am UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691/5 "2019-11-19T05:00:42Z")

</div>

@sebgl

Thank you for your advice.  
As your advice, i seperated my yaml file into 1) Elasticsearch search with default options and 2) Service for the LB settings like you wrote above.

And, i finally can use Elasticsearch on the LB and kibana.

Thank you again for your kind advice.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 4, 2022, 7:32am UTC](https://discuss.elastic.co/t/kibana-cant-connect-to-es-having-openstack-loadbalancer-setting/206691/6 "2022-11-04T07:32:16Z")

</div>