Sporadic unable to authenticate user OIDC errors

[Eugene_Marcotte]

Hey there,

I've been working on setting up a elastic cloud hosted deployment (currently running v7.8.0) with OIDC pointed at Okta. I've run into this error sporadically with various users:

"[security_exception] unable to authenticate user [<OIDC Token>] for action [cluster:admin/xpack/security/oidc/authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"

Following along with OpenID error after authenticating against AWS Cognito I have OIDC trace enabled. I find that if I end up not getting the error I see trace logs showing my token and user info responses and am able to log in and everyone is happy. When I do get the error I do not get any trace logs.

Am a bit at a loss for next steps to debug here. Any pointers would be awesome.

This is the relevant es.yaml segment:

xpack.security.authc.realms.oidc.okta:
  order: 2
  rp.client_id: "okta client id"
  rp.response_type: code
  rp.redirect_uri: "https://kibana-cloud-domain/api/security/v1/oidc"
  rp.requested_scopes: [openid, email, profile, groups]
  op.issuer: "https://our-domain-here.okta.com"
  op.authorization_endpoint: "https://our-domain-here.okta.com/oauth2/v1/authorize"
  op.token_endpoint: "https://our-domain-here.okta.com/oauth2/v1/token"
  op.jwkset_path: "https://our-domain-here.okta.com/oauth2/v1/keys"
  op.userinfo_endpoint: "https://our-domain-here.okta.com/oauth2/v1/userinfo"
  op.endsession_endpoint: "https://our-domain-here.okta.com/oauth2/v1/logout"
  rp.post_logout_redirect_uri: "https://kibana-cloud-domain/logged_out"
  claims.principal: sub
  claims.name: name
  claims.mail: email
  claims.groups: groups

Here's my kibana block:

xpack.security.authc.providers:
  oidc.okta:
    order: 1
    realm: okta
    description: "Log in with Okta"
    hint: "This is probably what you want!"
    icon: "image url here"
  
  basic.basic1:
    order: 2
    hint: "This is for super admins"

PS. I also sometimes get what looks like a javascript error on the login screen, clicking the option a second time a few seconds later usually lets the request go through. Not sure if it's related.

[ikakavas (Ioannis Kakavas)]

If you can enable debug logging for kibana and also capture a HAR from your browser when the error happens, we will be more than happy to take a look and see what the issue might be.

PS. I also sometimes get what looks like a javascript error on the login screen,

You mean in Kibana? Can you share the exact error ?

[Eugene_Marcotte]

For the JS error: yes, in kibana. Reproducing it now in a private window, I realized just now that it is also trying to report an unauthorized response. Here's what it reports...

Header of the dialog: Could not perform login.
Error field: Unauthorized

Stack trace

_construct@https://kibana-cloud.tom.takeoff.com/31997/bundles/core/core.entry.js:26:56491
Wrapper@https://kibana-cloud.tom.takeoff.com/31997/bundles/core/core.entry.js:26:55881
_createSuper/<@https://kibana-cloud.tom.takeoff.com/31997/bundles/core/core.entry.js:26:54669
HttpFetchError@https://kibana-cloud.tom.takeoff.com/31997/bundles/core/core.entry.js:26:57648
_callee3$@https://kibana-cloud.tom.takeoff.com/31997/bundles/core/core.entry.js:46:149749
l@https://kibana-cloud.tom.takeoff.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:288:969217
s/o._invoke</<@https://kibana-cloud.tom.takeoff.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:288:968971
_/</e[t]@https://kibana-cloud.tom.takeoff.com/31997/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:288:969574
asyncGeneratorStep@https://kibana-cloud.tom.takeoff.com/31997/bundles/core/core.entry.js:46:143272
_next@https://kibana-cloud.tom.takeoff.com/31997/bundles/core/core.entry.js:46:143601

I clicked the login button again and it brought me to the okta login prompt.

As for debug logs, any particular category or just across the board for kibana? Will try to reproduce (it would be so much eaiser if it happened consistently ) and capture a HAR.

[Eugene_Marcotte]

While trying to generate the HAR I hit the same sort of pop up on the main page saying an error occurred, in the respond on the network tab it had

message: "[security_exception] Cannot find OpenID Connect realm with name [okta]"

Is it possible that some of the instances are out of sync somehow?

[Eugene_Marcotte]

I may have found a solution. The problem is likely me not understanding that all the different types of ES instances have individual elasticsearch.yaml entry fields in the deployment UI, and even if it says no significant changes when copy-pasting config between them, they are using independent config or something . Can you tell this is my first time using ES as administrator?

[ikakavas (Ioannis Kakavas)]

Thanks for the feedback Eugene. We will try to make this more clear in our documentation!