Authentication problem using OIDC

[firabby]

I am not sure where to post this. As I am seeing this error in kibana, I am posting here.

I am using 30 day on premise trial license to test a project where I need to use OIDC. My elasticsearch.yml is:

xpack.security.enabled: true
xpack.security.http.ssl.enabled: false
xpack.security.authc.token.enabled: true
xpack.security.authc.realms.oidc.oidc1:
      order: 2
      rp.client_id: "client_id"
      rp.response_type: code
      rp.redirect_uri: "http://localhost:5601/api/security/v1/oidc"
      op.issuer: "issuer_url"
      op.authorization_endpoint: "auth_endpoint"
      op.token_endpoint: "token_endpoint"
      op.jwkset_path: "jwks_url"
      op.userinfo_endpoint: "userinfo_url"
      rp.post_logout_redirect_uri: "http://localhost:5601/logged_out"
      claims.principal: sub
      claims.groups: ""

and I have OIDC client secret in keystore. I also generated passwords for built in users, i.e. kibana, apm_user etc. But I only changed password for kibana user in kibana.yml.

My kibana,yml looks like:

server.host: "localhost"
elasticsearch.username: "kibana"
elasticsearch.password: "password"
elasticsearch.ssl.verificationMode: none
xpack.security.authc.providers: [oidc, basic]
xpack.security.authc.oidc.realm: oidc1
server.xsrf.whitelist: [/api/security/v1/oidc]
logging.dest: /var/log/kibana/kibana.log

When I try to login to kibana, the kibana redirects to OIDC provider login page. I login to the OIDC provider and I am redirected to kibana server where I see the following message:

{
"statusCode": 401,
"error": "Unauthorized",
"message": "[security_exception] unable to authenticate user [<OIDC Token>] for action [cluster:admin/xpack/security/oidc/authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"
}

How do I solve this problem. Thanks in advance.

[Brandon_Kobel]

Hey @firabby, would you mind enabling trace logging for OIDC using the following:

PUT /_cluster/settings
{
  "transient": {
    "logger.org.elasticsearch.xpack.security.authc.oidc": "trace"
  }
}

trying to reauthenticate, and checking your Elasticsearch logs?

[firabby]

I enabled logging and there was an error regarding ssl certificate. so, I generated ssl certificates and tried authentication. Following is the log output:

[2019-12-19T15:26:16,067][INFO ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication of [kibana] was terminated by realm [reserved] - failed to authenticate user [kibana]
[2019-12-19T15:26:16,069][INFO ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication of [kibana] was terminated by realm [reserved] - failed to authenticate user [kibana]
[2019-12-19T15:26:16,386][INFO ][o.e.c.r.a.AllocationService] [X556UQK] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[security-auditlog-2019.12.04][0]]]).
[2019-12-19T15:26:16,812][INFO ][o.e.c.m.MetaDataIndexTemplateService] [X556UQK] adding template [.management-beats] for index patterns [.management-beats]
[2019-12-19T15:26:19,923][WARN ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to get user information from the UserInfo endpoint.]; nested: IllegalStateException[Error merging ID token and userinfo claim value for claim [email_verified]. Cannot merge [java.lang.Boolean] with [java.lang.String]]
[2019-12-19T15:26:21,231][WARN ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to get user information from the UserInfo endpoint.]; nested: IllegalStateException[Error merging ID token and userinfo claim value for claim [email_verified]. Cannot merge [java.lang.Boolean] with [java.lang.String]]
[2019-12-19T15:27:53,209][WARN ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to get user information from the UserInfo endpoint.]; nested: IllegalStateException[Error merging ID token and userinfo claim value for claim [email_verified]. Cannot merge [java.lang.Boolean] with [java.lang.String]]
[2019-12-19T15:28:02,427][WARN ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to get user information from the UserInfo endpoint.]; nested: IllegalStateException[Error merging ID token and userinfo claim value for claim [email_verified]. Cannot merge [java.lang.Boolean] with [java.lang.String]]
[2019-12-19T15:28:08,899][WARN ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to get user information from the UserInfo endpoint.]; nested: IllegalStateException[Error merging ID token and userinfo claim value for claim [email_verified]. Cannot merge [java.lang.Boolean] with [java.lang.String]]
[2019-12-19T15:28:10,099][WARN ][o.e.x.s.a.AuthenticationService] [X556UQK] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to get user information from the UserInfo endpoint.]; nested: IllegalStateException[Error merging ID token and userinfo claim value for claim [email_verified]. Cannot merge [java.lang.Boolean] with [java.lang.String]]

[Brandon_Kobel]

For anyone else experiencing this same issue, the conversation continues in topic Failed to authenticate user with OpenID Connect