Getting unauthorised saml user error

I'm implementing SAML for elasticsearch cluster. After hitting kibana url I get redirected to IDP saml page for login but after login I get below error in browser -

< {"statusCode":401,"error":"Unauthorized","message":"[security_exception] unable to authenticate user [<unauthenticated-saml-user>] for action [cluster:admin/xpack/security/saml/authenticate], with { header={ WWW-Authenticate="Basic realm=\"security\" charset=\"UTF-8\"" } } :: {"path":"/_xpack/security/saml/authenticate","query":{},"body":"{\"ids\":[\"_d263c43dea4fa7997bddfa0eaf6fbbdb7ed11011\"]>

Seeing below error in elasticsearch.yml file:-
<[2019-04-12T09:57:43,977][WARN ][o.a.x.s.s.XMLSignature ] Signature verification failed.
[2019-04-12T09:57:43,997][WARN ][o.e.x.s.a.AuthenticationService] [es-coordinating] Authentication to realm saml1 failed - Provided SAML response is not valid for realm saml/saml1 (Caused by ElasticsearchSecurityException[SAML assertion [U8eyywQktnmBPxDobRY/fZisgzoe62kh...] is encrypted, but no decryption key is available])>

NOTE- I have confirmed that ( sp.entity_id ) match with what has been configured as the SAML Service Provider Entity ID in the SAML Identity Provider documentation

Elasticsearch configuration:-
xpack.security.authc.token.enabled: true
xpack.security.authc.realms.saml1:
type: saml
order: 2
idp.metadata.path: /usr/share/elasticsearch/config/saml/idp-metadata.xml
idp.entity_id: "XXXSSO"
sp.entity_id: "https://gbikibana-xxxx.corp.XXXX.com"
sp.acs: "https://gbikibana-xxxx.corp.XXXX.com:443/api/security/v1/saml"
sp.logout: "https://gbikibana-xxxx.corp.XXXX.com:443/logout"
attributes.principal: "nameid:persistent"

kibana config:-
server:
host: 0.0.0.0
xpack.security.public:
protocol: https
hostname: gbikibana-xxxx.corp.XXXX.com
port: 443
elasticsearch.url: "https://es-coordinating.gbi-xxxx.svc.lb.usrno1.XXXX.io:9200"
elasticsearch.username: kibana
elasticsearch.password: kibana
xpack.security.enabled: true
server.ssl.enabled: true
server.ssl.key: /usr/share/kibana/config/tls_server/key.pem
server.ssl.certificate: /usr/share/kibana/config/tls_server/crt.pem
elasticsearch.ssl.certificateAuthorities: /usr/share/kibana/config/tls_server/crt.pem
xpack.security.authProviders: [saml]
server.xsrf.whitelist: [/api/security/v1/saml]

Hi there

The error is actually very clearly pointed out in your logs:

Your SAML IDP is sending an encrypted SAML Assertion but you haven't configured Elasticsearch for this and it can't decrypt it.

Have you read our documentation? We have a very detailed saml guide and the encryption settings in particular are discussed in this part: Configure Elasticsearch for SAML authentication | Elasticsearch Guide [7.0] | Elastic

You can, but we discourage it. The signing & encryption certificates for SAML are unrelated to your server identity for network services (TLS), and it is not usually a good idea to mix them.
It will work, but you're better off just generating a new certificate.

You do, but your IdP team already thinks you have a certificate, so you should probably ask them about that before you go any further.

SAML assertion [U8eyywQktnmBPxDobRY/fZisgzoe62kh...] is encrypted,
  but no decryption key is available

If you are getting this error, then the IdP thinks it is supposed to send you encrypted assertions, and is encrypting them using some sort of certificate that it got from somewhere.

That could be a mistake from your IdP team - maybe they just copied the config from another service and forgot to remove the encryption certificate. Maybe they generated a new certificate for you, but didn't give you a copy.

It would be advisable for you to talk to them about this before you try and fix it from your side.

yes the IDP is using some certificate to encrypt responses. So I should provide them my own crt.pem and ask them to use the same. Just one question, I should provide crt.pem of es-coordinating or kibana ?

The question is why? and can they just turn it off.
You might have a local policy to always encrypt assertions, but the circumstances under which this is truly necessary are pretty rare. So for simplicity's sake, I would suggest you simply disable encryption (on the IdP) if you have that option.

No, as mentioned above, we recommend that you generate a new certificate specifically for SAML encryption and/or signing. The documentation that @ikakavas linked to (above) walks you through that process.
They don't need a copy of your TLS certificates, they just need your SAML encryption certificate (the encryption.certificate setting from your realm).

Alternatively, If your IdP can import SAML metadata, then you can simply generate a metadata file using the elasticsearch-saml-metadata tool, and pass that to your IdP team.

Hi TimV,
This worked after IDP disabled encryption from there end. As of now this is in UAT so disabling encryption is fine but once I move to production, is it mandatory to enable encryption from IDP end ? Any harm to Elasticsearch security if we keep encryption disabled ?

Not sure about any local policy to encrypt assertions

Elasticsearch will not impose any requirements on the encryption of SAML Assertions. For most of the cases, the necessary confidentiality property which is provided by TLS (over which the SAML Assertion is sent, via https ) is enough. If the IDP, or a local security policy dictates the use of encrypted SAML Assertions, Elasticsearch can consume them if correctly configured.

Yes, you do. By default your users have no roles, and as such no privilege to access anything, please see our documentation: Configuring role mappings | Elasticsearch Guide [7.0] | Elastic

Not sure what you mean by "it had routed me to kibana UI" but there is no way your SAML user would have access to any indices or the kibana UI (you need the kibana_user role) unless you have explicitly granted them the role.

@Ronnie16
It's best to open a new topic when you run into a differerent issue.

The error you are seeing is a Kibana matter, and you're going to get a better result in the kibana forum.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.