Not able to sign in using Keycloak in Kibana

Elastic Stack Kibana
1

Hi,

I have integrated ES, Kibana with Keycloak. Both Kibana and ES are https enabled. The below are the configurations I have made in the yml files.

## elasticsearch.yml

#Open ID connect settings
xpack.security.authc.realms.oidc.c9:
  enabled: true
  order: 2
  rp.client_id: "my-kibana-8"
  rp.response_type: code
  rp.redirect_uri: "https://<ip-address>:5601/api/security/oidc/callback"
  op.issuer: "https://<ip-address>:8443/realms/c9/"
  op.authorization_endpoint: "https://<ip-address>:8443/realms/c9/protocol/openid-connect/auth"
  op.token_endpoint: "https://<ip-address>:8443/realms/c9/protocol/openid-connect/token"
  op.jwkset_path: "https://<ip-address>:8443/realms/c9/protocol/openid-connect/certs"
  op.userinfo_endpoint: "https://<ip-address>:8443/realms/c9/protocol/openid-connect/userinfo"
  claims.principal: sub
  claims.mail: email
  claims.name: name
  ssl.verification_mode: none
  

##  kibana.yml


# OIDC
xpack.security.authc.providers:
  oidc.c9:
    order: 0
    realm: "c9"
    description: "Login with Keycloak"
  basic.basic1:
    order: 1

image
image700×545 18.4 KB

image
image670×658 38.3 KB

I get the below error when I enter the credentials

image
image591×579 26.7 KB

While in Keycloak it shows up in session

image
image1523×496 23.7 KB

Below are the settings I had made in Keycloak

image
image1457×646 46.9 KB

image
image1024×655 30 KB

image
image801×484 23.8 KB

image
image1249×743 41.1 KB

image
image1317×766 32.8 KB

I have also entered the client secret in bin/elasticsearch-keystore add xpack.security.authc.realms.oidc.oidc1.rp.client_secret.

I have also created a role, shiru in the client and assigned the same to a user (Keycloak).
I have also role mapped (role-mapping) a role shiru with unique name shiru (Kibana).

Elasticsearch logs

[2022-11-08T17:49:29,667][WARN ][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication to realm c9 failed - Failed to authenticate user wi              th OpenID Connect (Caused by org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token)
2

Could you please share more logs from the Elasticsearch node? There should be extra logs after the following line that give out details for the underlying cause.

You can also enable trace logging with

PUT _cluster/settings
{
  "transient": {
    "logger.org.elasticsearch.xpack.security.authc": "trace"
  }
}

and share the logs when the authentication fails.

1 Like
3

Kibana Logs 

{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2022-11-09T09:57:47.658+04:00","message":"Logging in with provider \"c9\" (oidc)","log":{"level":"INFO","logger":"plugins.security.routes"},"process":{"pid":746687},"trace":{"id":"68f02332b6b464401074171dd2a8e602"},"transaction":{"id":"ca10f434e783694a"}}
{"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.4.0"},"@timestamp":"2022-11-09T09:57:55.692+04:00","message":"Could not authenticate user with the existing session. Forcing logout.","log":{"level":"WARN","logger":"plugins.security.authentication"},"process":{"pid":746687},"trace":{"id":"b76e7101673b92137d272da8b3cb511f"},"transaction":{"id":"1bb7532787e6af99"}}
4

Elasticsearch logs 

[2022-11-09T10:04:09,315][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:admin/get]]
[2022-11-09T10:04:09,514][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Found authentication credentials [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] for principal [kibana_system] in request [rest request uri [/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip]]
[2022-11-09T10:04:09,515][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Checking token of type [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] against [4] realm(s)
[2022-11-09T10:04:09,515][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Trying to authenticate [kibana_system] using realm [reserved/reserved] with token [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] 
[2022-11-09T10:04:09,515][DEBUG][o.e.x.s.a.e.ReservedRealm] [node-1] realm [reserved] authenticated user [kibana_system], with roles [[kibana_system]] (cached)
[2022-11-09T10:04:09,515][DEBUG][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication of [kibana_system] using realm [reserved/reserved] with token [UsernamePasswordToken] was [AuthenticationResult{status=SUCCESS, value=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], message=null, exception=null}]
[2022-11-09T10:04:09,515][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Established authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] for request [rest request uri [/_nodes?filter_path=nodes.*.version%2Cnodes.*.http.publish_address%2Cnodes.*.ip]]
[2022-11-09T10:04:09,515][TRACE][o.e.x.s.a.s.SecondaryAuthenticator] [node-1] no secondary authentication credentials found (the [es-secondary-authorization] header is [null])
[2022-11-09T10:04:09,515][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [cluster:monitor/nodes/info]]
[2022-11-09T10:04:09,516][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [cluster:monitor/nodes/info[n]]]
[2022-11-09T10:04:09,738][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:admin/rollover]]
[2022-11-09T10:04:09,739][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:monitor/stats]]
[2022-11-09T10:04:09,739][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:admin/rollover]]
[2022-11-09T10:04:09,740][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:monitor/stats]]
[2022-11-09T10:04:09,740][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:monitor/stats[n]]]
[2022-11-09T10:04:09,740][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:monitor/stats[n]]]
[2022-11-09T10:04:09,740][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:admin/rollover]]
[2022-11-09T10:04:09,740][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:monitor/stats]]
[2022-11-09T10:04:09,740][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=_xpack,roles=[],fullName=null,email=null,metadata={}], realm={Realm[__attach.__attach] on Node[node-1]}, type=USER, metadata={}},type=INTERNAL]] in request [transport request action [indices:monitor/stats[n]]]
[2022-11-09T10:04:09,949][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/read/search]]
[2022-11-09T10:04:09,949][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/read/search]]
[2022-11-09T10:04:09,950][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/read/search[phase/query]]]
[2022-11-09T10:04:10,373][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Found authentication credentials [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] for principal [kibana_system] in request [rest request uri [/.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true]]
[2022-11-09T10:04:10,374][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Checking token of type [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] against [4] realm(s)
[2022-11-09T10:04:10,374][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Trying to authenticate [kibana_system] using realm [reserved/reserved] with token [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] 
[2022-11-09T10:04:10,374][DEBUG][o.e.x.s.a.e.ReservedRealm] [node-1] realm [reserved] authenticated user [kibana_system], with roles [[kibana_system]] (cached)
[2022-11-09T10:04:10,374][DEBUG][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication of [kibana_system] using realm [reserved/reserved] with token [UsernamePasswordToken] was [AuthenticationResult{status=SUCCESS, value=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], message=null, exception=null}]
[2022-11-09T10:04:10,374][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Established authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] for request [rest request uri [/.kibana_task_manager/_update_by_query?ignore_unavailable=true&refresh=true]]
[2022-11-09T10:04:10,374][TRACE][o.e.x.s.a.s.SecondaryAuthenticator] [node-1] no secondary authentication credentials found (the [es-secondary-authorization] header is [null])
[2022-11-09T10:04:10,375][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/write/update/byquery]]
[2022-11-09T10:04:10,375][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/read/search]]
[2022-11-09T10:04:10,376][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/read/search[phase/query]]]
[2022-11-09T10:04:10,382][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/read/scroll/clear]]
[2022-11-09T10:04:10,382][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username
5

Sometimes I get this error too... in elasticsearch.log

[2022-11-09T10:19:34,738][DEBUG][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication of [kibana_system] using realm [reserved/reserved] with token [UsernamePasswordToken] was [AuthenticationResult{status=SUCCESS, value=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], message=null, exception=null}]
[2022-11-09T10:19:34,738][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Established authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] for request [rest request uri [/_security/oidc/authenticate]]
[2022-11-09T10:19:34,738][TRACE][o.e.x.s.a.s.SecondaryAuthenticator] [node-1] no secondary authentication credentials found (the [es-secondary-authorization] header is [null])
[2022-11-09T10:19:34,739][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [cluster:admin/xpack/security/oidc/authenticate]]
[2022-11-09T10:19:34,739][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Checking token of type [org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectToken] against [4] realm(s)
[2022-11-09T10:19:34,739][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Trying to authenticate [<OIDC Token>] using realm [oidc/c9] with token [org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectToken]
[2022-11-09T10:19:34,739][TRACE][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] OpenID Connect Provider redirected user to [/api/security/oidc/callback?state=a_fmApO8eAkBxT2RWctYmQGX1plIrgGChsC6FsQ_nas&session_state=51b8c186-68b8-446a-a79b-21dc6775a5e6&code=6e7b7da2-275a-474e-9b56-edaddeeae65a.51b8c186-68b8-446a-a79b-21dc6775a5e6.48349f75-6240-4b93-866d-aafe06350a97]. Expected Nonce is [LY3S1BzOLVa6tDfZCCZeHo_2nii6G_QYORij9xHdaus] and expected State is [a_fmApO8eAkBxT2RWctYmQGX1plIrgGChsC6FsQ_nas]
[2022-11-09T10:19:34,772][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] effective HTTP connection keep-alive: [180000]ms
[2022-11-09T10:19:34,773][TRACE][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] Successfully exchanged code for ID Token [com.nimbusds.jwt.SignedJWT@17140b6] and Access Token [ey***2Q]
[2022-11-09T10:19:34,774][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] ID Token Header: {"kid":"3J9biLIREcVjhoVBdH3_gAmRsviLSy8v5VTMJLwNoys","typ":"JWT","alg":"RS256"}
[2022-11-09T10:19:34,780][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] effective HTTP connection keep-alive: [180000]ms
[2022-11-09T10:19:34,781][TRACE][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] Successfully refreshed and cached remote JWKSet
[2022-11-09T10:19:34,781][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] ID Token Header: {"kid":"3J9biLIREcVjhoVBdH3_gAmRsviLSy8v5VTMJLwNoys","typ":"JWT","alg":"RS256"}
[2022-11-09T10:19:34,782][DEBUG][o.e.x.s.a.o.OpenIdConnectRealm] [node-1] Failed to consume the OpenIdConnectToken
org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator.getUserClaims(OpenIdConnectAuthenticator.java:300) ~[?:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator.lambda$getUserClaims$1(OpenIdConnectAuthenticator.java:294) ~[?:?]
        at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:162) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:113) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.ListenableFuture.done(ListenableFuture.java:100) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.BaseFuture.set(BaseFuture.java:131) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.ListenableFuture.onResponse(ListenableFuture.java:139) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator$ReloadableJWKSource$1.completed(OpenIdConnectAuthenticator.java:985) ~[?:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator$ReloadableJWKSource$1.completed(OpenIdConnectAuthenticator.java:976) ~[?:?]
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) ~[?:?]
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182) ~[?:?]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) ~[?:?]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) ~[?:?]
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[?:?]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87) ~[?:?]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:121) ~[?:?]
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[?:?]
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[?:?]
        at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: com.nimbusds.jwt.proc.BadJWTException: Unexpected JWT issuer: https://10.46.0.18:8443/realms/c9
        at com.nimbusds.openid.connect.sdk.validators.IDTokenClaimsVerifier.verify(IDTokenClaimsVerifier.java:175) ~[?:?]
        at com.nimbusds.jwt.proc.DefaultJWTProcessor.verifyClaims(DefaultJWTProcessor.java:295) ~[?:?]
        at com.nimbusds.jwt.proc.DefaultJWTProcessor.process(DefaultJWTProcessor.java:400) ~[?:?]
        at com.nimbusds.openid.connect.sdk.validators.IDTokenValidator.validate(IDTokenValidator.java:288) ~[?:?]
        at com.nimbusds.openid.connect.sdk.validators.IDTokenValidator.validate(IDTokenValidator.java:224) ~[?:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator.getUserClaims(OpenIdConnectAuthenticator.java:267) ~[?:?]
        ... 23 more
[2022-11-09T10:19:34,785][DEBUG][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication of [<OIDC Token>] using realm [oidc/c9] with token [OpenIdConnectToken] was [AuthenticationResult{status=CONTINUE, value=null, message=Failed to authenticate user with OpenID Connect, exception=org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token}]
[2022-11-09T10:19:34,785][WARN ][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication to realm c9 failed - Failed to authenticate user with OpenID Connect (Caused by org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token)
[2022-11-09T10:19:34,785][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Failed to authenticate request [transport request action [cluster:admin/xpack/security/oidc/authenticate]]
[2022-11-09T10:19:34,790][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Found authentication credentials [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] for principal [kibana_system] in request [rest request uri [/.kibana_security_session/_doc/QY%2Fr1CyEXf9enclicd%2BmPZGN9x%2FhLUhFMJYnGCKpk6A%3D?refresh=wait_for]]
[2022-11-09T10:19:34,790][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Checking token of type [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] against [4] realm(s)
[2022-11-09T10:19:34,790][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Trying to authenticate [kibana_system] using realm [reserved/reserved] with token [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken]
[2022-11-09T10:19:34,790][DEBUG][o.e.x.s.a.e.ReservedRealm] [node-1] realm [reserved] authenticated user [kibana_system], with roles [[kibana_system]] (cached)
[2022-11-09T10:19:34,791][DEBUG][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication of [kibana_system] using realm [reserved/reserved] with token [UsernamePasswordToken] was [AuthenticationResult{status=SUCCESS, value=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], message=null, exception=null}]
[2022-11-09T10:19:34,791][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Established authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] for request [rest request uri [/.kibana_security_session/_doc/QY%2Fr1CyEXf9enclicd%2BmPZGN9x%2FhLUhFMJYnGCKpk6A%3D?refresh=wait_for]]
[2022-11-09T10:19:34,791][TRACE][o.e.x.s.a.s.SecondaryAuthenticator] [node-1] no secondary authentication credentials found (the [es-secondary-authorization] header is [null])
6

Thanks for sharing extra logs. Unfortunately, they do not contain useful information. We need logs that are around the original warnning that you shared

2022-11-08T17:49:29,667][WARN ][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication to realm c9 failed - Failed to authenticate user wi th OpenID Connect (Caused by org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token)

Please try locate a similar log message that is most recent and share any debug/trace logs around it (include itself as well). Thanks!

1 Like
7

The incoming logs changed a bit after executing the command you said

The log you have mentioned is after the blank space

[2022-11-09T10:26:09,844][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Trying to authenticate [<OIDC Token>] using realm [oidc/c9] with token [org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectToken]
[2022-11-09T10:26:09,845][TRACE][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] OpenID Connect Provider redirected user to [/api/security/oidc/callback?state=McUN5LFzzOjBqHtBRJ7Mz7WUPiGT_A_JLzaJvpqC7L0&session_state=b0a71ad3-147b-47a9-96e4-86262b26f0c8&code=12f60983-523d-4a11-aa1a-1f0d8f6579a5.b0a71ad3-147b-47a9-96e4-86262b26f0c8.48349f75-6240-4b93-866d-aafe06350a97]. Expected Nonce is [_RIDbmeYB_VRpuS46UeDAZMQ-APqr9adlzRKtsGWU2Y] and expected State is [McUN5LFzzOjBqHtBRJ7Mz7WUPiGT_A_JLzaJvpqC7L0]
[2022-11-09T10:26:09,872][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] effective HTTP connection keep-alive: [180000]ms
[2022-11-09T10:26:09,873][TRACE][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] Successfully exchanged code for ID Token [com.nimbusds.jwt.SignedJWT@a04509] and Access Token [ey***eQ]
[2022-11-09T10:26:09,874][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] ID Token Header: {"kid":"3J9biLIREcVjhoVBdH3_gAmRsviLSy8v5VTMJLwNoys","typ":"JWT","alg":"RS256"}
[2022-11-09T10:26:09,891][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] effective HTTP connection keep-alive: [180000]ms
[2022-11-09T10:26:09,892][TRACE][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] Successfully refreshed and cached remote JWKSet
[2022-11-09T10:26:09,892][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [node-1] ID Token Header: {"kid":"3J9biLIREcVjhoVBdH3_gAmRsviLSy8v5VTMJLwNoys","typ":"JWT","alg":"RS256"}
[2022-11-09T10:26:09,892][DEBUG][o.e.x.s.a.o.OpenIdConnectRealm] [node-1] Failed to consume the OpenIdConnectToken
org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator.getUserClaims(OpenIdConnectAuthenticator.java:300) ~[?:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator.lambda$getUserClaims$1(OpenIdConnectAuthenticator.java:294) ~[?:?]
        at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:162) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:113) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.ListenableFuture.done(ListenableFuture.java:100) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.BaseFuture.set(BaseFuture.java:131) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.common.util.concurrent.ListenableFuture.onResponse(ListenableFuture.java:139) ~[elasticsearch-8.5.0.jar:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator$ReloadableJWKSource$1.completed(OpenIdConnectAuthenticator.java:985) ~[?:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator$ReloadableJWKSource$1.completed(OpenIdConnectAuthenticator.java:976) ~[?:?]
        at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) ~[?:?]
        at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182) ~[?:?]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) ~[?:?]
        at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) ~[?:?]
        at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[?:?]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87) ~[?:?]
        at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:121) ~[?:?]
        at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[?:?]
        at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[?:?]
        at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[?:?]
        at java.lang.Thread.run(Thread.java:1589) ~[?:?]
Caused by: com.nimbusds.jwt.proc.BadJWTException: Unexpected JWT issuer: https://10.46.0.18:8443/realms/c9
        at com.nimbusds.openid.connect.sdk.validators.IDTokenClaimsVerifier.verify(IDTokenClaimsVerifier.java:175) ~[?:?]
        at com.nimbusds.jwt.proc.DefaultJWTProcessor.verifyClaims(DefaultJWTProcessor.java:295) ~[?:?]
        at com.nimbusds.jwt.proc.DefaultJWTProcessor.process(DefaultJWTProcessor.java:400) ~[?:?]
        at com.nimbusds.openid.connect.sdk.validators.IDTokenValidator.validate(IDTokenValidator.java:288) ~[?:?]
        at com.nimbusds.openid.connect.sdk.validators.IDTokenValidator.validate(IDTokenValidator.java:224) ~[?:?]
        at org.elasticsearch.xpack.security.authc.oidc.OpenIdConnectAuthenticator.getUserClaims(OpenIdConnectAuthenticator.java:267) ~[?:?]
        ... 23 more
[2022-11-09T10:26:09,896][DEBUG][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication of [<OIDC Token>] using realm [oidc/c9] with token [OpenIdConnectToken] was [AuthenticationResult{status=CONTINUE, value=null, message=Failed to authenticate user with OpenID Connect, exception=org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token}]





[2022-11-09T10:26:09,896][WARN ][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication to realm c9 failed - Failed to authenticate user with OpenID Connect (Caused by org.elasticsearch.ElasticsearchSecurityException: Failed to parse or validate the ID Token)
[2022-11-09T10:26:09,896][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Failed to authenticate request [transport request action [cluster:admin/xpack/security/oidc/authenticate]]
[2022-11-09T10:26:09,901][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Found authentication credentials [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] for principal [kibana_system] in request [rest request uri [/.kibana_security_session/_doc/GKlXbFEecQSUglMlrgZBS3iIE81UvFfxd5jMkjHas6g%3D?refresh=wait_for]]
[2022-11-09T10:26:09,901][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Checking token of type [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken] against [4] realm(s)
[2022-11-09T10:26:09,901][TRACE][o.e.x.s.a.RealmsAuthenticator] [node-1] Trying to authenticate [kibana_system] using realm [reserved/reserved] with token [org.elasticsearch.xpack.core.security.authc.support.UsernamePasswordToken]
[2022-11-09T10:26:09,901][DEBUG][o.e.x.s.a.e.ReservedRealm] [node-1] realm [reserved] authenticated user [kibana_system], with roles [[kibana_system]] (cached)
[2022-11-09T10:26:09,901][DEBUG][o.e.x.s.a.RealmsAuthenticator] [node-1] Authentication of [kibana_system] using realm [reserved/reserved] with token [UsernamePasswordToken] was [AuthenticationResult{status=SUCCESS, value=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], message=null, exception=null}]
[2022-11-09T10:26:09,901][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Established authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] for request [rest request uri [/.kibana_security_session/_doc/GKlXbFEecQSUglMlrgZBS3iIE81UvFfxd5jMkjHas6g%3D?refresh=wait_for]]
[2022-11-09T10:26:09,901][TRACE][o.e.x.s.a.s.SecondaryAuthenticator] [node-1] no secondary authentication credentials found (the [es-secondary-authorization] header is [null])
[2022-11-09T10:26:09,902][TRACE][o.e.x.s.a.AuthenticatorChain] [node-1] Found existing authentication [Authentication[effectiveSubject=Subject{version=8.5.0, user=User[username=kibana_system,roles=[kibana_system],fullName=null,email=null,metadata={_reserved=true}], realm={Realm[reserved.reserved] on Node[node-1]}, type=USER, metadata={}},type=REALM]] in request [transport request action [indices:data/write/delete]]
8

OK. The last log has this error message

Unexpected JWT issuer: https://10.46.0.18:8443/realms/c9

Is this the same value that you have configured in elasticsearch.yml, i.e.:

op.issuer: "https://<ip-address>:8443/realms/c9/"

I think there is at least one difference which is the trailing slash character (/).

1 Like
9

Thanks,

that worked.

I had given the / becoz somewhere in the documentation I saw the issuer url should be such that it should give OIDC configurations when .well-known/openid-configuration is attached to it.

10

Can you tell me what this is for?

Will it affect my data?

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.