Kibana 6.8.2 deploying on K8S ( using security xpack feature )

Elastic Stack Kibana
,
1

Hi everyone ,

My name is Larry , I've been using ELK stack for 2 years and I'm enjoying a lot this platform . I've earned a lot of useful experience , that helps me to ease all procedures of logging for developers .

Not so long ago I decided to deploy my Logstash and Kibana services on Kubernetes , but then I've been caught by a little problem .

Problem : I want to use 2 pods ( to provide load balancing ) of Kibana with the security feature , but when I try to log in it redirects me to a "Log In" page without any "errors".

I'm using images of Logstash 6.8.2 , Kibana 6.8.2 and Elastic cluster is distributed on VMs , all the stack worked perfect , but then I decided to add xpack security feature and found out that I can't use 2 pods of Kibana in the same Deployment at the same moment . After that I tried to use only 1 pod and it works as it supposed to work , I also checked presence of conflicts between VM + container ... there is no problem , tried to add configuration of session affinity in ClusterIP service and it didn't help . I guess that the problem is in my K8S configuration and I'm close to success , but it's not enough .

Thank you for all the support ! I hope I'm not at the dead end and I'll be able to solve my problem with your help :heart:

P.S.: If there is no solution I'm glad to get feedback about your best practice of working with ELK on K8S .

2

Can you explain in detail what happens ? When you try to login at which of the two Kibanas ? Does one work and the other doesn't ? Do you enter your username and password and just get redirected to the login page again without any errors ? What is written in the logs ?

When you use multiple kibana instances talking to the same elasticsearch cluster, you need to make sure that xpack.security.encryptionKey in kibana.yml is set to the same value in all your kibana instances. Otherwise, when you authenticate with kibana 1 and then at some point your LB sends you to kibana 2, it will not be able to read your session cookie and treat you as unauthenticated user.

Have you looked into our ECK ?

3

Thank you for your attention , Ioannis .

I configured Elastic with xpack.security, elastic-setup-passwords and gave Kibana its elastic.username and elastic.password ( checked that VM works and 1 pod in K8S works ) and the only problem is 2 pods in K8S .

I connect with the Ingress URL to Kibana service ( I don't really know which pod is used ) and after entering the username and password I'm redirected to a clear "Log In" page to enter my username and password again without getting error of "wrong username / password " or something , and in pods I get logs of :

{"type":"response","@timestamp":"2019-08-28T12:01:57Z","tags":[],"pid":1,"method":"get","statusCode":200,"req":{"url":"/login?next=%2Fstatus","method":"get","headers":{"host":"<***my pod IP***>:5601","user-agent":"kube-probe/1.13","referer":"http://<***my pod IP***>:5601/status","accept-encoding":"gzip","connection":"close"},"remoteAddress":"<***my remote address***>","userAgent":"<***my remote agent ( same as address )***>","referer":"http://<***my pod IP***>:5601/status"},"res":{"statusCode":200,"responseTime":9,"contentLength":9},"message":"GET /login?next=%2Fstatus 200 9ms - 9.0B"}

I'll check the ECK , thanks . It also can help me to know your opinion if it's worth to use 2 ( or more ) pods to LoadBalance the Kibana , or 1 pod is enough to get a good performance .

4

Greetings , Ioannis !

I just wanted to say , that the problem was really with cookies and the line of xpack.security.encryptionKey solved this .

Thank you very much , I couldn't reply you earlier to say how much you helped me :smile:

1 Like
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.