Is Kibana supported on GKE through default google ingress ? doesn't seem so

I have deployed Elastic Search and Kibana on GKE.
However unable to get ingress working as it seems to report Kibana service as unhealthy.
I assume this is due to the root url redirecting with http 302 and GKE LB L7 expecting status of 200 always based on reading around.
Can someone confirm this and if there is a work around provide as I have spent quiet a bit of trying to get it working and seem to be stuck with the whole thing.

Hi @rubans. Kibana is definitely supported on GKE. Have you seen our Elastic Cloud on Kubernetes documentation?

Or, if you prefer a lower level implementation, we also publish Helm charts for the Elastic Stack.

These options should be much easier than trying to create and deploy your own CRD.

Appreciate the quick reply.
Unfortunately Helm charts is not an option for a corporate install I’m doing but it’s not a bad idea to understand the components anyway.
However at present I can’t get the ingress service working as mentioned and judging by some of the posts seems an issue.
Can you comment on the specific ingress issue ?

@nickpeihl Looks like GCE LB health checks in GCP always expect a 200 and it's not possible to change this, can someone respond ?
If there is a fix for this already in chart packages then can you provide on the nature of the fix as I'm not sure it has been fixed.

Hi @rubans. I don't know enough about Kubernetes to answer further. I suggest re-posting your question in the Elastic Orchestration category. Hopefully someone there can help.

Hi @nickpeihl, it's specifically a Kibana issue even if related to Orchestration so I think the forum is relevant.
I find the whole Elastic support overly complicated a bit like the platform, I have started looking at other options in the mean time due to the lack of clear support for the product.

Hi @rubans

Apologies for your struggles, read this and wanted to use this as a learning experience for myself.

I am certainly not a K8s expert etc. but I got the ngnix-ingress controller to work just fine on GKE, I am sure I am missing some nuances somewhere.

Where is How I got GKE + ECK + ngnix-ingress-controller + TLS working.

This is what I did after a created a Vanilla GKE cluster

Of course I did all the role binding stuff etc.. etc..

I am using ECK so I applied that...

kubectl apply -f https://download.elastic.co/downloads/eck/1.4.0/all-in-one.yaml

Then per here I applied

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.44.0/deploy/static/provider/cloud/deploy.yaml

I created a cert, dns etc... etc..

Then I I applied this

kubectl apply -f es-kb-ingress.yaml

This is es-kb-ingress.yaml

apiVersion: elasticsearch.k8s.elastic.co/v1beta1
kind: Elasticsearch
metadata:
  name: d3m0
spec:
  version: 7.11.1
  nodeSets:
  - name: default
    count: 3
    config:
      node.store.allow_mmap: false
---
apiVersion: kibana.k8s.elastic.co/v1beta1
kind: Kibana
metadata:
  name: d3m0
spec:
  version: 7.11.1
  count: 1
  elasticsearchRef:
    name: d3m0
  http:
    tls:
      selfSignedCertificate:
        disabled: true       
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: kibana-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  tls:
    - hosts:
      - kib.mydomain.net
      # Assumes Realcert with above host in it
      secretName: my-kb-cert
  rules:
  - host: kib.mydomain.net
    http:
      paths:
      - path: /
        backend:
          serviceName: d3m0-kb-http
          servicePort: 5601

And this is what I got a nice secured stack from an ingress perspective (I would secure the back end as well)

Perhaps give it a try...

I suspect I am missing something, you asked for default... hmmm I will try that next

1 Like

Hi @stephenb
This is really useful. Appreciate you taking the time to create an end to end example.
I actually managed to resolve my issue yesterday with a simple fix.
My issue was down to an error in the ingress configuration.
I originally tried the default GKE ingress which I believe is not supported with Elastic stack (specifically Kibana UI)
In that I had some wildcard paths "/*" which I copied over straight to the nginx ingress, I wasn't aware that the wild card path syntax in the same manner aren't supported with nginx ingress controller. So this led to some weird issues such repeated redirects (302).
I once I swapped to path value "/", similar to what you have in your example, it all started playing nicely.

Nice!

I am halfway through with the default... but now I have a path problem LOL. I am sure I will get it sorted, perhaps I am running into the same thing... yeah just noticed the syntax slightly different between default and ngnix... hey I always like ngnix anyways!

@stephenb
Let me know if you need any additional info, not that I'm now an expert :slightly_smiling_face: or anything.
One of the issues I had is that I'm behind a heavy corporate security so I couldn't just deploy the ECK or Helm charts as it is from Elastic.
I had to download and decipher the charts as I had to rebuild the various components docker images from source then apply them. So just meant, I was never sure whether it was me doing something wrong or something inherently not supported. Anyway, really appreciate you taking time to go the extra mile to assist. Always appreciated.

1 Like

@rubans
Just FYI, I got everything to work on my side for default ingress as well. I totally understand the "Corporate Sercurity " I lived with them for many years at a financial institute. I do think that occasionally there is a race condition with setting some times while ELK / Kibana is coming up and the ingress is trying to come up. I had better luck letting ECK / ELK come all the way up and then applying the ingress... if I have time I will do another one of my little How Tos.. Thanks for the co-adventure!

@stephenb do you have your deployed final conigs?
Especially interested in the ingress config to see the reason why I couldn't get it working.
Also, I raised an issue with the latest Helm Chart and ingress package which also exhibited the same issue.