Kubernetes: how to run HA kibana behind service? - Login fails if using more than one instance


I am running elastic stack with enabled security module (TLS + authentication) in kubernetes.
My Infrastructure inside kubernetes looks like this:

I have one issue and one question:

If running only one kibana pod, Login is running fine, I can work with kibana. But If I run 2 pods behind the kibana service, I the Login window comes back after entering credentials. I am caught in a loop.

So what do I need to to if I want to use multiple kibana instances behind a service / loadbalancer?
I think I read somewhere of a parameter / key / cookie stuff which should be configured identically on all kibana instances which are behind a loadbalancer. But I am not too sure about it and I cannot find this information again.

Is that picture shown best practice for running kibana + elasticsearch in kubernetes or is it still better to run kibana against it's own coordinating only node as described in production recommendations?

Thanks a lot,

PS: I did not set any session affinity yet.

You need to set xpack.security.encryptionKey to be the same in all of your Kibana instances as by default this will be auto-generated with a different value every time Kibana starts. See https://www.elastic.co/guide/en/kibana/current/security-settings-kb.html#security-ui-settings

great, that was the link I lost :wink:

I just added xpack.security.encryptionKey to keystore and it works again. Many thanks.

Can anyone please also answer the Best practice question above?