# How Elasticsearch does verify JWT tokens? **URL:** \ **Category:** Elasticsearch\ **Tags:** elastic-stack-security\ **Created:** [June 17, 2024, 12:29pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581 "2024-06-17T12:29:49Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Post date:** [June 17, 2024, 12:29pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/1 "2024-06-17T12:29:49Z")
Hello, I have 6 node Elasticsearch cluster in acceptance enviroment. I have 2 master, 2 hot and 2 warm nodes. I want to implement authentication via Keycloak. In elasticsearch.yml I configured. ```auto xpack.security.authc.token.enabled: true xpack.security.authc.realms: oidc: oidc1: order: 2 rp.client_id: "elasticsearch" rp.response_type: "code" rp.redirect_uri: "https://apkibana.-***/api/security/oidc/callback" rp.post_logout_redirect_uri: "https://apkibana.-***/logged_out" op.issuer: "https://keycloak.-***/realms/vnttest" op.authorization_endpoint: "https://keycloak.-***/realms/vnttest/protocol/openid-connect/auth" op.token_endpoint: "https://keycloak.-***/realms/vnttest/protocol/openid-connect/token" op.jwkset_path: "https://keycloak.-***/realms/vnttest/protocol/openid-connect/certs" op.userinfo_endpoint: "https://keycloak.-***/realms/vnttest/protocol/openid-connect/userinfo" op.endsession_endpoint: "https://keycloak.-***/realms/vnttest/protocol/openid-connect/logout" #claims.principal: "preferred_username" claims.principal: "sub" claims.groups: "roles" ``` in kibana.yml I configured ```auto xpack.security.authc.providers: oidc.oidc1: order: 0 realm: oidc1 description: "Keycloak" basic.basic: order: 1 ``` Redirect from Kibana to Keycloak is working, then I should get token and access Kibana but I always fail to permissions. I have no permission to access. When I try to debug with curl: ```auto curl --location --request POST 'https://keycloak.- ***/realms/vnttest/protocol/openid-connect/token' --header 'Content-Type: application/x-www-form-urlencoded' --data-urlencode 'grant_type=password' --data-urlencode 'client_id=elasticsearch' --data-urlencode 'client_secret=****' --data-urlencode 'username= ****' --data-urlencode 'password=*** ``` I get token: `{"access_token":" *******************","expires_in":300,"refresh_expires_in":1800,"refresh_token":"*************","token_type":"Bearer","not-before-policy":0,"session_state":"cce2 ****** 3370","scope":"email profile groups"}` But when I try to authenticate: ```auto curl -X GET "https://es-node1***:9200/_xpack/security/_authenticate" -H "Authorization: Bearer "Token" ``` I got: ```auto {"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate with provided credentials and anonymous access is not allowed for this request","additional_unsuccessful_credentials":"oauth2 token: invalid token","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}}],"type":"security_exception","reason":"unable to authenticate with provided credentials and anonymous access is not allowed for this request","additional_unsuccessful_credentials":"oauth2 token: invalid token","header":{"WWW-Authenticate":["Basic realm=\"security\" charset=\"UTF-8\"","Bearer realm=\"security\"","ApiKey"]}},"status":401} ``` Any advice? There is nothing in logs. ES log is clear and kibana logs: `"Logging in with provider \"oidc1\" (oidc)","log":{"level":"INFO","logger":"plugins.security.routes"},"process":{"pid":605362,"uptime":66536.670301688},"trace":{"id":"42cc8bfc351db5f7b37d71a848d16869"},"transaction":{"id":"e87196424eba00cd"}}` When should I debug? Why I cant auth via command line? from JWT token I can see that I got: ```auto "exp": 1718625334, "iat": 1718625034, "jti": "b936c1 **** 9064cb37", "iss": "https://keycloak- ***** realms/vnttest", "aud": [ "realm-management", "broker", "account" ], "sub": "ffd8e08 ***** 1051c205e", "typ": "Bearer", "azp": "elasticsearch", "session_state": "d29261 **** ebb04f7a375", "acr": "1", "allowed-origins": [ "*", "https://apkibana- ****" ], "realm_access": { "roles": [ "elasticsearch", "offline_access", "uma_authorization", "vntadmin", "default-roles-vnttest" ] }, "resource_access": { "realm-management": { "roles": [ "view-realm", "view-identity-providers", "manage-identity-providers", "impersonation", "realm-admin", "create-client", "manage-users", "query-realms", "view-authorization", "query-clients", "query-users", "manage-events", "manage-realm", "view-events", "view-users", "view-clients", "manage-authorization", "manage-clients", "query-groups" ] }, "elasticsearch": { "roles": [ "vntadmin" ] }, "broker": { "roles": [ "read-token" ] }, "account": { "roles": [ "manage-account", "view-applications", "view-consent", "view-groups", "manage-account-links", "manage-consent", "delete-account", "view-profile" ] } }, "scope": "email profile groups", "sid": "d292 ***** 7a375", "email_verified": false, "roles": [ "vntadmin" ], "name": " *****", "preferred_username": "vnttest", "given_name": " ****", "family_name": " ****", "email": " ****" } ``` In Kibana I set role\_mapping: ```auto PUT /_security/role_mapping/keycloak_role_mapping { "roles": ["superuser"], "enabled": true, "rules": { "all": [ { "field": { "realm_access.roles": "vntadmin" } }, { "field": { "resource_access.elasticsearch.roles": "vntadmin" } } ] } } ``` ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [June 17, 2024, 12:33pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/2 "2024-06-17T12:33:12Z")
Is the license correct? You need at least a platinum or enterprise license. ---
**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Post date:** [June 17, 2024, 1:16pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/3 "2024-06-17T13:16:36Z")
Yes. ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/f/3f14f2139fe0687eb84b91b4495889314432f16a.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 17, 2024, 2:55pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/4 "2024-06-17T14:55:07Z")
Hi @vnovotny98 I am Not a JWT Exper,but from the [docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/jwt-auth-realm.html#hmac-oidc-example-request-headers) on request headers perhaps you need the `-H "ES-Client-Authentication: SharedSecret client-shared-secret-string"` as well > You can use this header in a `curl` request to make an authenticated call to Elasticsearch. Both the bearer token and the client authorization token must be specified as separate headers with the `-H` option: > > `curl -s -X GET -H "Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJpc3M4IiwiYXVkIjoiYXVkOCIsInN1YiI6InNlY3VyaXR5X3Rlc3RfdXNlciIsImV4cCI6NDA3MDkwODgwMCwiaWF0Ijo5NDY2ODQ4MDB9.UnnFmsoFKfNmKMsVoDQmKI_3-j95PCaKdgqqau3jPMY" -H "ES-Client-Authentication: SharedSecret client-shared-secret-string" https://localhost:9200/_security/_authenticate` ---
**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Post date:** [June 17, 2024, 4:07pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/5 "2024-06-17T16:07:26Z")
Hi, I tried by your tips. I also set: ```auto logger.org.elasticsearch.xpack.security.authc: DEBUG logger.org.elasticsearch.xpack.security.authc.token: DEBUG logger.org.elasticsearch.xpack.security.authc.support: DEBUG ``` I tried with different APIs, with different users. But everytime I got this error: ```auto [2024-06-17T18:03:08,060][DEBUG][o.e.x.s.a.TokenService] [xxxx-xxxx] built in token service unable to decode token java.io.IOException: Illegal base64 character 0x2e at java.util.Base64$DecInputStream.read(Base64.java:1163) ~[?:?] at org.elasticsearch.core.Streams.readFully(Streams.java:129) ~[elasticsearch-core-8.13.4.jar:?] at org.elasticsearch.common.io.stream.InputStreamStreamInput.readBytes(InputStreamStreamInput.java:55) ~[elasticsearch-8.13.4.jar:?] at org.elasticsearch.common.io.stream.StreamInput.readByteArray(StreamInput.java:982) ~[elasticsearch-8.13.4.jar:?] at org.elasticsearch.xpack.security.authc.TokenService.decodeToken(TokenService.java:672) ~[?:?] at org.elasticsearch.xpack.security.authc.TokenService.tryAuthenticateToken(TokenService.java:474) ~[?:?] at org.elasticsearch.xpack.security.authc.OAuth2TokenAuthenticator.doAuthenticate(OAuth2TokenAuthenticator.java:73) ~[?:?] at org.elasticsearch.xpack.security.authc.OAuth2TokenAuthenticator.authenticate(OAuth2TokenAuthenticator.java:69) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$getAuthenticatorConsumer$5(AuthenticatorChain.java:167) ~[?:?] at org.elasticsearch.xpack.core.common.IteratingActionListener.onResponse(IteratingActionListener.java:135) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticatorChain.lambda$getAuthenticatorConsumer$5(AuthenticatorChain.java:146) ~[?:?] at org.elasticsearch.xpack.core.common.IteratingActionListener.run(IteratingActionListener.java:117) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticatorChain.doAuthenticate(AuthenticatorChain.java:125) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticate(AuthenticatorChain.java:95) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:264) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:152) ~[?:?] at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:130) ~[?:?] at org.elasticsearch.xpack.security.Security.lambda$getHttpTransports$35(Security.java:1758) ~[?:?] at org.elasticsearch.xpack.security.Security.lambda$getHttpServerTransportWithHeadersValidator$38(Security.java:1815) ~[?:?] at org.elasticsearch.http.netty4.internal.HttpHeadersAuthenticatorUtils.lambda$getValidatorInboundHandler$2(HttpHeadersAuthenticatorUtils.java:47) ~[?:?] at org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.lambda$requestStart$1(Netty4HttpHeaderValidator.java:139) ~[?:?] at org.elasticsearch.action.ActionListener.run(ActionListener.java:356) ~[elasticsearch-8.13.4.jar:?] at org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.requestStart(Netty4HttpHeaderValidator.java:113) ~[?:?] at org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.channelRead(Netty4HttpHeaderValidator.java:61) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?] at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?] at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1383) ~[?:?] at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1246) ~[?:?] at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1295) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[?:?] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[?:?] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[?:?] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[?:?] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[?:?] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[?:?] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) ~[?:?] at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689) ~[?:?] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652) ~[?:?] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) ~[?:?] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[?:?] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[?:?] at java.lang.Thread.run(Thread.java:1583) ~[?:?] [2024-06-17T18:03:08,062][DEBUG][o.e.x.s.a.AuthenticatorChain] [xxx-xxxx] Authenticating with null credentials is unsuccessful in request [rest request uri [/_xpack/security/_authenticate]] after unsuccessful attempts of other credentials ``` I tried: ```auto curl -X GET "https://xxx-xxx:9200/_security/_authenticate" -H "ES-Client-Authentication: SharedSecret 0qUPuF ****mzvjG" -H "Authorization: Bearer eyJhbGciOiJSUzI1N***** Hq-Sf-T-lHyYWotGyWL4k_g" curl -X GET "https://xxx-xxx:9200/_xpack/security/_authenticate" -H "ES-Client-Authentication: SharedSecret 0qUPuF ****mzvjG" -H "Authorization: Bearer eyJhbGciOiJSUzI1N***** Hq-Sf-T-lHyYWotGyWL4k_g" curl -X GET "https://xxx-xxx:9200/security/_authenticate" -H "ES-Client-Authentication: SharedSecret 0qUPuF ****mzvjG" -H "Authorization: Bearer eyJhbGciOiJSUzI1N***** Hq-Sf-T-lHyYWotGyWL4k_g" ``` with or without client auth, but still didnt resolve the problem ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 17, 2024, 4:40pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/6 "2024-06-17T16:40:54Z")
> [@vnovotny98](#): > > `Illegal base64 character 0x2e` Did you base64 encode the the token? ---
**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Post date:** [June 17, 2024, 5:07pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/7 "2024-06-17T17:07:44Z")
I got the Token by: ```auto curl --location --request POST 'https://keycloak. **** /realms/vnttest/protocol/openid-connect/token' \ --header 'Content-Type: application/x-www-form-urlencoded' \ --data-urlencode 'grant_type=password' \ --data-urlencode 'client_id=elasticsearch' \ --data-urlencode 'client_secret=0qUPxxxxmzvjG' \ --data-urlencode 'username=xxxx' \ --data-urlencode 'password=xxx' ``` By this I got the JSON Web Token (JWT) from Keycloak. But not enrypted by base64. I think that Elasticsearch should be able to decode it right? I tried to decode this bye [jwt.io](http://jwt.io) and it was successfull so token is alright. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 17, 2024, 7:11pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/8 "2024-06-17T19:11:31Z")
did you try with single quotes `curl -X GET 'https://xxx-xxx:9200/_security/_authenticate' -H 'ES-Client-Authentication: SharedSecret 0qUPuF ****mzvjG' -H 'Authorization: Bearer eyJhbGciOiJSUzI1N***** Hq-Sf-T-lHyYWotGyWL4k_g'` ---
**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Post date:** [June 17, 2024, 8:48pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/9 "2024-06-17T20:48:30Z")
I tried now, but didnt help. In every documentation there is double quota also. I think that my ELK deploy cant run Bearer tokens, but I dont know why. I think from what I read that everything is set. I have same xpack config on every node. For the tests I try to auth only to one node where I run commands. I think that this error displays because some missing configuration. ```auto built in token service unable to decode token java.io.IOException: Illegal base64 character 0x2e ``` I cant find some step by step instructions how to connect ELK to Keycloak 😕 ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 17, 2024, 9:09pm UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/10 "2024-06-17T21:09:21Z")
Have you enabled a platinum license? > [@vnovotny98](#): > > For the tests I try to auth only to one node where I run commands. How do you know you are only sending to one node? its complaining about the `.` ...`character 0x2e` > **[Keycloak Authentication for ELK Stack](https://blog.devops.dev/keycloak-authentication-for-elk-stack-900dedb8b27a)** > > Youtube Video ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [June 18, 2024, 1:14am UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/11 "2024-06-18T01:14:02Z")
> [@vnovotny98](#): > > ```auto > xpack.security.authc.realms: > oidc: > oidc1: > > ``` You don't have a JWT realm, you have an OIDC realm. The OIDC realm is designed to work using Browser SSO, it is not supported from the command line. > [@vnovotny98](#): > > Redirect from Kibana to Keycloak is working, then I should get token and access Kibana but I always fail to permissions. I have no permission to access. If you are trying to make this work in Kibana, then please provide the messages you get when trying to authenticate with Kibana. You've gone down a rabbit hole of trying to make it work with curl, but it doesn't sound like that's your goal here. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 18, 2024, 1:18am UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/12 "2024-06-18T01:18:22Z")
Thanks @TimV.. clearly the a piece I was missing... ---
**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Post date:** [June 18, 2024, 7:35am UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/13 "2024-06-18T07:35:11Z")
You are right @TimV I always got in kibana to: ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/a/2a0f545a0a47ae6c7fb2298182b083bb1f8040bb.png) kibana log ```auto {"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2024-06-18T09:16:04.570+02:00","message":"Complete. Task run took 6ms [stated: 2024-06-18T07:16:04.564Z]","log":{"level":"INFO","logger":"p lugins.securitySolution.endpoint:user-artifact-packager:1.0.0"},"process":{"pid":1238,"uptime":85533.059046819},"trace":{"id":"290230c583ec233637dcb1ac120a15a1"},"transaction":{"id":"560b8b6a2a1abd6d"}} {"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2024-06-18T09:16:57.624+02:00","message":"Logging in with provider \"oidc1\" (oidc)","log":{"level":"INFO","logger":"plugins.security.routes"},"process":{"pid":1238,"uptime":85586.113404401},"trace":{"id":"87a129a418490b51143ba3e9cae5ca64"},"transaction":{"id":"eea2d60d2f4dca6d"}} {"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2024-06-18T09:17:07.536+02:00","message":"Started. Checking for changes to endpoint artifacts","log":{"level":"INFO","logger":"plugins.securitySolution.endpoint:user-artifact-packager:1.0.0"},"process":{"pid":1238,"uptime":85596.024589437},"trace":{"id":"81e22d03036ce217edc95c794fbdd39f"},"transaction":{"id":"4acebe5e32393d98"}} {"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2024-06-18T09:17:07.541+02:00","message":"Last computed manifest not available yet","log":{"level":"INFO","logger":"plugins.securitySolution.endpoint:user-artifact-packager:1.0.0"},"process":{"pid":1238,"uptime":85596.03032864},"trace":{"id":"81e22d03036ce217edc95c794fbdd39f"},"transaction":{"id":"4acebe5e32393d98"}} {"service":{"node":{"roles":["background_tasks","ui"]}},"ecs":{"version":"8.11.0"},"@timestamp":"2024-06-18T09:17:07.542+02:00","message":"Complete. Task run took 6ms [stated: 2024-06-18T07:17:07.536Z]","log":{"level":"INFO","logger":"plugins.securitySolution.endpoint:user-artifact-packager:1.0.0"},"process":{"pid":1238,"uptime":85596.030600862},"trace":{"id":"81e22d03036ce217edc95c794fbdd39f"},"transaction":{"id":"4acebe5e32393d98"}} ``` elasticsearch debug log ```auto 2024-06-18T09:22:37,186][DEBUG][o.e.x.s.a.s.m.NativeRoleMappingStore] [nodemaster-XXX] Mapping user [UserData{username:ffd8e081-7cfc-48c3-a068-9501051c205e; dn:null; groups:[vntadmin]; metadata:{oidc(preferred_username)=vnttest, oidc(sid)=d296bed6-7a1f-4820-85cd-23b9d86752d2, oidc(family_name)=XXX, oidc(at_hash)=NzjdlxW1XXXXJrZYZxg, oidc(iss)=https://keycloak.XXXX/realms/vnttest, oidc(name)=Vojtech Novotny, oidc(id_token_hint)=eyJhbGciOiJSUzI1XXXXX7hsga6gMg2m8x6GcKoh_sVkLjCuA2X1i36XElfk3rtH_4MCR1ql6gF_aLND3-w, oidc(jti)=4e1a9076-XXXXXd0bd38, oidc(session_state)=d296bed6-XXXX52d2, oidc(nonce)=xMTrhgxzheMHdUA5f3WgXXXXA655zE, oidc(email_verified)=false, oidc(aud)=[elasticsearch], oidc(auth_time)=1718695357, oidc(given_name)=Vojtech, oidc(typ)=ID, oidc(roles)=["vntadmin"], oidc(email)=vnovotXXX, oidc(sub)=ffd8e08XXXXc205e, oidc(azp)=elasticsearch, oidc(acr)=1}; realm=oidc1}] to roles [[]] [2024-06-18T09:22:37,186][DEBUG][o.e.x.s.a.RealmsAuthenticator] [apesmaster01-nhq] Authentication of [] using realm [oidc/oidc1] with token [OpenIdConnectToken] was [AuthenticationResult{status=SUCCESS, value=User[username=ffdXXXX1051c205e,roles=[],fullName=null,email=null,metadata={oidc(preferred_username)=vnttest, oidc(sid)=d296bed6-7XXXX6752d2, oidc(family_name)=Novotny, oidc(at_hash)=NzjdlxWXXXZYZxg, oidc(iss)=https://keycloakXXXX/realms/vnttest, oidc(name)=Vojtech Novotny, oidc(id_token_hint)=eyJhbXXXXLND3-w, oidc(jti)=4e1a907XXXXd0bd38, oidc(session_state)=d296beXXX52d2, oidc(nonce)=xMTrhgxXXXX655zE, oidc(email_verified)=false, oidc(aud)=[elasticsearch], oidc(auth_time)=1718695357, oidc(given_name)=Vojtech, oidc(typ)=ID, oidc(roles)=["vntadmin"], oidc(email)=vnovXXXX, oidc(sub)=ffd8e081-7XXXX5e, oidc(azp)=elasticsearch, oidc(acr)=1}], message=null, exception=null}] [2024-06-18T09:22:37,187][DEBUG][o.e.x.s.a.TokenService] [apesmaster01-nhq] Using refresh policy [NONE] when creating token doc [token_4YlErs4QXXXfRua-2zBvCuE] in the security index [.security-tokens] ``` I wanted to try out what am I missing so I wanted to test it by curl but now I understand. So I think that problem is in role mapping? Can you help me set role\_mapping that will fix it for me? I tried: ```auto PUT /_security/role_mapping/keycloak_role_mapping { "roles": ["superuser"], "enabled": true, "rules": { "all": [ { "field": { "elasticsearch.roles": "vntadmin" } }, { "field": { "resource_access.elasticsearch.roles": "vntadmin" } }, { "field": { "roles": "vntadmin" } } ] } } ``` With set in elasticsearch: ```auto claims.principal: "sub" claims.groups: "roles" ``` ---
**Author:** ![vnovotny98](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vnovotny98/32/124809_2.png) [@vnovotny98](https://discuss.elastic.co/u/vnovotny98)\ **Post date:** [June 18, 2024, 11:16am UTC](https://discuss.elastic.co/t/how-elasticsearch-does-verify-jwt-tokens/361581/14 "2024-06-18T11:16:04Z")
I resolved it. With setting: ``` claims.principal: "preferred_username" claims.groups: "roles" ``` ```auto PUT /_security/role_mapping/keycloak_role_mapping { "roles": ["own_pgw_viewer"], "enabled": true, "rules": { "any": [ { "field": { "metadata.oidc(roles)": "vntadmin" } } ] } } ``` So the User field is **metadata.oidc(roles)** Then answer from ELK is: ```auto [2024-06-18T13:09:36,783][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [XXX-xxx] effective HTTP connection keep-alive: [180000]ms [2024-06-18T13:09:36,785][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [XXX-xxx] ID Token Header: {"kid":"6YXXXXXAk","typ":"JWT","alg":"RS256"} [2024-06-18T13:09:36,794][DEBUG][o.e.x.s.a.o.OpenIdConnectAuthenticator] [XXX-xxx] effective HTTP connection keep-alive: [180000]ms [2024-06-18T13:09:36,800][DEBUG][o.e.x.s.a.s.m.NativeRoleMappingStore] [XXX-xxx] successfully loaded [1] role-mapping(s) from [.security] [2024-06-18T13:09:36,800][DEBUG][o.e.x.s.a.s.m.NativeRoleMappingStore] [XXX-xxx] Mapping user [UserData{username:vnttest; dn:null; groups:[vntadmin]; metadata:{oidc(email)=XXX-xxx, oidc(roles)=["vntadmin"], oidc(typ)=ID, oidc(given_name)=XXX-xxx, oidc(auth_time)=1718708976, oidc(aud)=[elasticsearch], oidc(email_verified)=false, oidc(nonce)=XXX-xxx, oidc(session_state)=4XXX-xxx, oidc(jti)=XXX-xxx, oidc(id_token_hint)=eyJhbGciOiXXX-xxxw, oidc(name)=XXX-xxx, oidc(iss)=https://keycloakXXX-xxx/realms/vnttest, oidc(at_hash)=FWYhXXX-xxxL8ZQ, oidc(family_name)=NXXX-xxx, oidc(sid)=45741eXXX-xxx37, oidc(preferred_username)=vnttest, oidc(acr)=1, oidc(azp)=elasticsearch, oidc(sub)=ffd8e0XXX-xxx5e}; realm=oidc1}] to roles [[own_pgw_viewer]] [2024-06-18T13:09:36,801][DEBUG][o.e.x.s.a.RealmsAuthenticator] [XXX-xxx] Authentication of [] using realm [oidc/oidc1] with token [OpenIdConnectToken] was [AuthenticationResult{status=SUCCESS, value=User[username=vnttest,roles=[own_pgw_viewer],fullName=null,email=null,metadata={oidc(email)=vXXX-xxx, oidc(roles)=["vntadmin"], oidc(typ)=ID, oidc(given_name)=VXXX-xxx, oidc(auth_time)=1718708976, oidc(aud)=[elasticsearch], oidc(email_verified)=false, oidc(nonce)=2A2e6XXX-xxx, oidc(jti)=XXX-xxx, oidc(id_token_hint)=eyXXX-xxx, oidc(name)=XXX-xxx, oidc(iss)=https://keycloak.XXX-xxx/realms/vnttest, oidc(at_hash)=FWXXX-xxx oidc(family_name)=XXX-xxx, oidc(sid)=4574XXX-xxx, oidc(preferred_username)=vnttest, oidc(acr)=1, oidc(azp)=elasticsearch, oidc(sub)=ffXXX-xxx}], message=null, exception=null}] [2024-06-18T13:09:36,801][DEBUG][o.e.x.s.a.TokenService] [XXX-xxx] Using refresh policy [NONE] when creating token doc [token_dsAvk0hANCKkJcALvSGb5rwxHC5MP0fr2fD4obfyYb4] in the security index [.security-tokens] ``` Thank you!