Elastic OpenID Keyclock failed cannot find realm

[Dave_Hafid]

hi guys i need help, we try to configure sso using keycloak , using OpenID connect we facing issue and i dont know how to fix this

and this error

[2024-02-29T12:37:27,257][WARN ][r.suppressed             ] [elastic01] path: /_security/oidc/prepare, params: {}, status: 500
org.elasticsearch.ElasticsearchSecurityException: Cannot find OpenID Connect realm with name [kpi]
        at org.elasticsearch.xpack.security.action.oidc.TransportOpenIdConnectPrepareAuthenticationAction.doExecute(TransportOpenIdConnectPrepareAuthenticationAction.java:89) ~[?:?]
        at org.elasticsearch.xpack.security.action.oidc.TransportOpenIdConnectPrepareAuthenticationAction.doExecute(TransportOpenIdConnectPrepareAuthenticationAction.java:28) ~[?:?]
        at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:87) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:85) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:163) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$DelegatingFailureActionListener.onResponse(ActionListenerImplementations.java:212) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$8(AuthorizationService.java:455) ~[?:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:1028) ~[?:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:994) ~[?:?]
        at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:32) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$9(AuthorizationService.java:469) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:236) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authz.RBACEngine.authorizeClusterAction(RBACEngine.java:186) ~[?:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:459) ~[?:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:435) ~[?:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:322) ~[?:?]
        at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:178) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:32) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:151) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:236) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$4(CompositeRolesStore.java:194) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:236) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRole$5(CompositeRolesStore.java:212) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:236) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$0(RoleReferenceIntersection.java:49) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:236) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:56) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromRoleReference(CompositeRolesStore.java:292) ~[?:?]
        at org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$1(RoleReferenceIntersection.java:53) ~[?:?]
        at java.lang.Iterable.forEach(Iterable.java:75) ~[?:?]
        at org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.buildRole(RoleReferenceIntersection.java:53) ~[?:?]
        at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:210) ~[?:?]
        at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:187) ~[?:?]
        at org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:147) ~[?:?]
        at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:338) ~[?:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:159) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:236) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:95) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticate(AuthenticatorChain.java:93) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:262) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:171) ~[?:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:155) ~[?:?]
        at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:114) ~[?:?]
        at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:85) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:62) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:196) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:108) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.client.internal.node.NodeClient.doExecute(NodeClient.java:86) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:381) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.rest.action.oidc.RestOpenIdConnectPrepareAuthenticationAction.lambda$innerPrepareRequest$0(RestOpenIdConnectPrepareAuthenticationAction.java:64) ~[?:?]
        at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:103) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.rest.SecurityRestFilter.doHandleRequest(SecurityRestFilter.java:94) ~[?:?]
        at org.elasticsearch.xpack.security.rest.SecurityRestFilter.lambda$handleRequest$0(SecurityRestFilter.java:85) ~[?:?]
        at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:178) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.lambda$authenticateAndAttachToContext$3(SecondaryAuthenticator.java:99) ~[?:?]
        at org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:236) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticate(SecondaryAuthenticator.java:109) ~[?:?]
        at org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticateAndAttachToContext(SecondaryAuthenticator.java:90) ~[?:?]
        at org.elasticsearch.xpack.security.rest.SecurityRestFilter.handleRequest(SecurityRestFilter.java:79) ~[?:?]
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:441) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:570) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:325) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:458) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:554) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:431) ~[elasticsearch-8.12.1.jar:?]
        at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.handlePipelinedRequest(Netty4HttpPipeliningHandler.java:128) ~[?:?]
        at org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:118) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
        at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?]
        at org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.forwardData(Netty4HttpHeaderValidator.java:194) ~[?:?]
        at org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.forwardFullRequest(Netty4HttpHeaderValidator.java:137) ~[?:?]
        at org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.lambda$requestStart$1(Netty4HttpHeaderValidator.java:120) ~[?:?]
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[?:?]
        at io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:106) ~[?:?]
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[?:?]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[?:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[?:?]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566) ~[?:?]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?]
        at java.lang.Thread.run(Thread.java:1583) ~[?:?]

my configuration

xpack.security.authc.realms.oidc.oidc1:
  order: 2
  rp.client_id: "kibana"
  rp.response_type: code
  rp.redirect_uri: "https://kibana01.korelasi.local:5601/api/security/oidc/callback"
  op.issuer: "https://10.199.199.45:8443/realms/kpi"
  op.authorization_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/auth"
  op.token_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/token"
  op.jwkset_path: "/etc/elasticsearch/certs/jwkset.json"
  op.userinfo_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/userinfo"
  op.endsession_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/logout"
  rp.post_logout_redirect_uri: "https://kibana01.korelasi.local:5601/security/logged_out"
  claims.principal: name
  ssl.verification_mode: none

kibana

xpack.security.authc.providers:
   oidc.oidc1:
     order: 0
     realm: kpi
   basic.basic1:
     order: 1

[stephenb (Stephen Brown)]

Hi @Dave_Hafid

Assuming you're following the docs and the examples on

Configuring single sign-on to the Elastic Stack using OpenID Connect |...

Authentication in Kibana | Kibana Guide [8.12] | Elastic

And your definition in elastic.yml

Seems like your realm is oidc1

@Dave_Hafid

realm: kpi

Not sure where kpi came from

Seems like it should be

realm: oidc1

Perhaps I'm missing something

[Dave_Hafid]

hi step thanks for you support its working now but still facing some issue

and this log error

[2024-02-29T14:48:02,106][WARN ][o.e.x.s.a.RealmsAuthenticator] [elastic01] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token)

and this is my realm setting

and this one

i need help

[stephenb (Stephen Brown)]

Unfortunately, I / we are not experts on Keycloak portion

did you try setting the realm name in keycloak to oidc1 as it is defined in elasticsearch.yml perhaps they need to be consistent

You can up the logging with

 PUT /_cluster/settings
 {
   "transient": {
     "logger.org.elasticsearch.xpack.security.authc.oidc": "trace"
   }
 }

There is also some advice here

Unable to configure oidc

we are trying to setup elasticsearch to use oidc for authentication here is my yaml file xpack.security.authc.token.enabled: true xpack.security.authc.realms.oidc.oidc1: order: 1 rp.client_id: "6bd9f9a3-67a3-4392-b29c-675c16b818e9" rp.response_type: "code" rp.redirect_uri: "https://<Kibana-Host>/api/security/oidc/callback" op.issuer: "https://<Login-Provider>" op.authorization_endpoint: "https://<Login-Provider>/oauth2/v2.0/authorize" op.t…

Authentication failed for an OpenID integration

Hello, I am using OIDC with Elasticsearch and Kibana. My configuration is as follows: xpack.security.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12 xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: elastic-stack-ca.p12 xpack.security.http.ssl.truststore.path: …

Post some of the additional logs and perhaps we can help.

I also notice that you elasticsearch.yml definitions do not follow the same patterns as in our example...

xpack.security.authc.realms.oidc.oidc1:
  order: 2
  rp.client_id: "the_client_id"
  rp.response_type: code
  rp.redirect_uri: "https://kibana.example.org:5601/api/security/oidc/callback"
  op.issuer: "https://op.example.org"
  op.authorization_endpoint: "https://op.example.org/oauth2/v1/authorize"
  op.token_endpoint: "https://op.example.org/oauth2/v1/token"
  op.jwkset_path: oidc/jwkset.json
  op.userinfo_endpoint: "https://op.example.org/oauth2/v1/userinfo"
  op.endsession_endpoint: "https://op.example.org/oauth2/v1/logout"
  rp.post_logout_redirect_uri: "https://kibana.example.org:5601/security/logged_out"
  claims.principal: sub
  claims.groups: "http://example.info/claims/groups"`

I am not an expert on that portion but what you have does not seem to follow the patterns but perhaps you are more of the expert

Example from here

[Dave_Hafid]

its working now, i put full path of jwkset.
thanks for your time and support

[stephenb (Stephen Brown)]

@Dave_Hafid
Thanks for letting us know. It works. That helps other people...

Can you provide a copy of your working config that can be very helpful for other people.

Thanks so much!

[Dave_Hafid]

this my elastic configuration

xpack.security.authc.realms.oidc.oidc1:
   order: 2
   rp.client_id: "kibana"
   rp.response_type: code
   rp.redirect_uri: "https://kibana01.korelasi.local:5601/api/security/oidc/callback"
   op.issuer: "https://10.199.199.45:8443/realms/kpi"
   op.authorization_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/auth"
   op.token_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/token"
   op.jwkset_path: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/certs"
   op.userinfo_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/userinfo"
   op.endsession_endpoint: "https://10.199.199.45:8443/realms/kpi/protocol/openid-connect/logout"
   rp.post_logout_redirect_uri: "https://kibana01.korelasi.local:5601/security/logged_out"
   claims.principal: name
   ssl.verification_mode: none

and my kibana configuration

xpack.security.authc.providers:
   oidc.oidc1:
     order: 0
     realm: oidc1
   basic.basic1:
     order: 1