Kibana user login backoff and redirect to login page again and again

Elastic Stack Kibana
1

Just setup Elasticsearch 7.1.0 to leverage user authentification. Cluster is setup with xpack security enabled and SSL in between nodes. I setup kibana with the kibana user and password, and enable xpack security as well. But I have this issue where when I log in with the elastic user, or any user I created via the _security API, Kibana seems to log the user in, I can see the Kibana homepage for a quick moment and I get kick out a few sec after, redirect to /logout and back to the login screen., with out any error message, not even in Kibana logs or in any elasticsearch nodes..

I need to mention that my whole stack is running under kubernetes, is there somthing to do with k8s service layer that do no redirect headers correctly, does Kibana look at the source IP and can not see it since it only see the k8s service ip ?

PRfbWJM5lx.gif1675×865

Thanks for your help :smiley:
Dave

2

Hmm, not sure exactly what's happening here. Have you tried clearing your cookies?

3

Yes, even try in incognito, on chrome, on safari and on another system, always same behavior

4

If this can help, this is the log from Kibana when I try to log in

{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/built_assets/css/plugins/infra/index.light.css","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"5d3bb2b0453688b6a9eb758344d70db4f6333576-/-gzip\""},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /built_assets/css/plugins/infra/index.light.css 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/built_assets/css/plugins/index_lifecycle_management/index.light.css","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"103e524f50b788471409c6986e6ea5dae8d0fd66-/-gzip\""},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /built_assets/css/plugins/index_lifecycle_management/index.light.css 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/built_assets/css/plugins/license_management/index.light.css","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"eb1fd43bca33019b0c410f729c251eee9d25b799-/-gzip\""},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /built_assets/css/plugins/license_management/index.light.css 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/built_assets/css/plugins/index_management/index.light.css","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"edc7799b372a708926207162516f1d0a7490f2da-/-gzip\""},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /built_assets/css/plugins/index_management/index.light.css 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/built_assets/css/plugins/maps/index.light.css","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"0f9f741abc5af302c2d61faaebed2e5f1bdf3e13-/-gzip\""},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /built_assets/css/plugins/maps/index.light.css 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/built_assets/css/plugins/ml/index.light.css","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"fe4bd8b7b728abb8c5ede7f65108572518c636c9-/-gzip\""},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /built_assets/css/plugins/ml/index.light.css 304 1ms - 9.0B"}
5 
{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/built_assets/css/plugins/spaces/index.light.css","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"text/css,*/*;q=0.1","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"c1baa218e783a9e6630d5046027bd77b892e30bf-/-gzip\""},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":0,"contentLength":9},"message":"GET /built_assets/css/plugins/spaces/index.light.css 304 0ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:31Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/ui/fonts/inter_ui/Inter-UI-Regular.woff2","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","origin":"http://100.64.127.59:32601","if-none-match":"\"5b94f1d60aba41c740a36bfb95f8f4aa77c91525\"","if-modified-since":"Thu, 16 May 2019 01:09:53 GMT","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"*/*","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7"},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":0,"contentLength":9},"message":"GET /ui/fonts/inter_ui/Inter-UI-Regular.woff2 304 0ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:32Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/ui/fonts/inter_ui/Inter-UI-SemiBold.woff2","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","origin":"http://100.64.127.59:32601","if-none-match":"\"13b5df4a7ab19ac1ce12b5d85e0e0a00340cb67b\"","if-modified-since":"Thu, 16 May 2019 01:09:53 GMT","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"*/*","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7"},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/login?next=%2Fapp%2Fkibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /ui/fonts/inter_ui/Inter-UI-SemiBold.woff2 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2019-05-22T19:47:32Z","tags":[],"pid":1,"method":"get","statusCode":304,"req":{"url":"/ui/images/bg_bottom_branded.svg","method":"get","headers":{"host":"100.64.127.59:32601","connection":"keep-alive","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36","accept":"image/webp,image/apng,image/*,*/*;q=0.8","referer":"http://100.64.127.59:32601/built_assets/css/plugins/security/index.light.css","accept-encoding":"gzip, deflate","accept-language":"en-CA,en;q=0.9,fr-CA;q=0.8,fr;q=0.7","if-none-match":"\"92c3a30f7ce7dbebb400a4711af5621d2f422f72-gzip\"","if-modified-since":"Thu, 16 May 2019 01:09:52 GMT"},"remoteAddress":"10.42.56.0","userAgent":"10.42.56.0","referer":"http://100.64.127.59:32601/built_assets/css/plugins/security/index.light.css"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /ui/images/bg_bottom_branded.svg 304 1ms - 9.0B"}
6

Any other thoughts ? :tired_face:

7

Hey I just found my solution, it was a k8s service issue, need to add a sessionaffinity

sessionAffinity: ClientIP

:slight_smile: GD

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.