Elasticsearch SAML2.0 and ADFS - Signature validation issue with single logout

Hello,

I am trying to implement a federation server for testing purposes. Currently, the applications and the users are in the same domain.

I will try to explain the issue I ran into and what I've troubleshot so far as precisely as possible.

Identity provider : AD FS

Service providers :

  • Elasticsearch/Kibana (both installed on the same machine, and there's only default sample data on elastic since I'm only interested in the authentication process)

I manage to log into Kibana (SP-initiated and IdP-initiated).
When I try to log out from Kibana, it works fine but only when I do it from the Kibana interface. But it doesn't work when I try to log out using AD FS.

The error I get with an IdP-initated logout is an Error 500 : Internal Server Error.

I looked into the Elastic logs with TRACE enabled. The issue seems to be related to signature validation :
[2019-07-15T13:49:53,228][WARN ][o.e.x.s.a.s.SamlLogoutRequestHandler] [node-1] The XML Signature of this SAML message cannot be validated. Please verify that the saml realm uses the correct SAMLmetadata file/URL for this Identity Provider

I double checked the certificates on both sides and they seem fine to me.

Here are the logs when I try to log in to Kibana (SP-initiated), then logout from Kibana (IdP-initiated) (TRACE logs omitted).

------------------ LOGIN SSO ------------------------

[2019-07-15T13:47:22,486][INFO ][o.o.x.k.i.p.InlineX509DataProvider] [node-1] The X509Data contained no X509Certificate elements, skipping credential extraction
[2019-07-15T13:47:22,496][WARN ][o.o.x.k.i.BasicProviderKeyInfoCredentialResolver] [node-1] No credentials could be extracted from KeyInfo child with QName {http://www.w3.org/2000/09/xmldsig#}X509Data by any registered provider
[2019-07-15T13:47:22,655][DEBUG][o.e.x.s.a.s.SamlAuthenticator] [node-1] SAML Signature [9u1SS160qrppWCs1fzFmZyfvxG6Q4RGc...] matches credentials [http://fs.iam.lab/adfs/services/trust]

[2019-07-15T13:47:22,671][DEBUG][o.e.x.s.a.s.SamlRealm    ] [node-1] Parsed token [SamlToken{3c73616d6c703a526573706f6e73652049443d225f38343266366135392d313437652d343730632d383561302d63616262636632313631323922205665727369...}] to attributes [SamlAttributes(NameId(null)=Administrator)[_4e361407-d2c1-4309-9856-c39ecf44bd49]{[]}]


----------------------- LOGOUT SSO ----------------------------

[2019-07-15T13:49:53,225][DEBUG][o.e.x.s.a.s.SamlLogoutRequestHandler] [node-1] SAML Signature [axo03uTNr2344nWNI3gmdvbbc+SfCByf] failed against credentials [http://fs.iam.lab/adfs/services/trust] 
[2019-07-15T13:49:53,227][DEBUG][o.e.x.s.a.s.SamlLogoutRequestHandler] [node-1] SAML Signature [axo03uTNr2344nWNI3gmdvbbc+SfCByf] failed against credentials [http://fs.iam.lab/adfs/services/trust]
[2019-07-15T13:49:53,228][WARN ][o.e.x.s.a.s.SamlLogoutRequestHandler] [node-1] The XML Signature of this SAML message cannot be validated. Please verify that the saml realm uses the correct SAMLmetadata file/URL for this Identity Provider
[2019-07-15T13:49:53,229][INFO ][o.e.x.s.a.s.TransportSamlInvalidateSessionAction] [node-1] Failed to invalidate SAML session
org.elasticsearch.ElasticsearchSecurityException: SAML Signature [axo03uTNr2344nWNI3gmdvbbc+SfCByf] could not be validated against [MIIDMTCCAhmgAwIBAgIIP3b3uW+4RU8wDQYJKoZIhvcNAQELBQAwHjEcMBoGA1UE...,MIIC0DCCAbigAwIBAgIQIlVuniwQ8L1F3pYttLc4MjANBgkqhkiG9w0BAQsFADAk...]
        at org.elasticsearch.xpack.security.authc.saml.SamlUtils.samlException(SamlUtils.java:105) ~[x-pack-security-7.1.1.jar:7.1.1]
        at org.elasticsearch.xpack.security.authc.saml.SamlRequestHandler.samlSignatureException(SamlRequestHandler.java:195) ~[x-pack-security-7.1.1.jar:7.1.1]
        at org.elasticsearch.xpack.security.authc.saml.SamlRequestHandler.checkIdpSignature(SamlRequestHandler.java:177) ~[x-pack-security-7.1.1.jar:7.1.1]
        at org.elasticsearch.xpack.security.authc.saml.SamlLogoutRequestHandler.validateSignature(SamlLogoutRequestHandler.java:118) ~[x-pack-security-7.1.1.jar:7.1.1]
(Stack trace truncated)

There's a few things that I don't understand there.
The first one if the INFO and WARN logs :

[2019-07-15T13:47:22,486][INFO ][o.o.x.k.i.p.InlineX509DataProvider] [node-1] The X509Data contained no X509Certificate elements, skipping credential extraction
[2019-07-15T13:47:22,496][WARN ][o.o.x.k.i.BasicProviderKeyInfoCredentialResolver] [node-1] No credentials could be extracted from KeyInfo child with QName {http://www.w3.org/2000/09/xmldsig#}X509Data by any registered provider

I don't know what they mean, I didn't find that much information on those. I don't think they're the source of my issue but I may be wrong (they appear during the authentication process). Could you please tell me more about those logs and what they mean ?

Also, I noticed that during the logout process, Elastic tries to validate the signature axo03uTNr2344nWNI3gmdvbbc+SfCByf against the certificates MIIDMTCCAhmgAwIBAgIIP3b3uW+4RU8wDQYJKoZIhvcNAQELBQAwHjEcMBoGA1UE... and MIIC0DCCAbigAwIBAgIQIlVuniwQ8L1F3pYttLc4MjANBgkqhkiG9w0BAQsFADAk....
It seems weird because the signature is way longer that 32 characters, but it doesn't seem to be truncated like the certificates values. So I supposed that Elastic was misreading the certificate value, would that be possible ?

Finally, I read that there could be compatibility issues if the IdP and SP did not use the same URL-encoding functions.

"Further, note that URL-encoding is not canonical; that is, there are multiple legal encodings for a given value. The relying party MUST therefore perform the verification step using the original URL-encoded values it received on the query string. It is not sufficient to re-encode the parameters after they have been processed by software because the resulting encoding may not match the signer's encoding."

Source : https://docs.oasis-open.org/security/saml/v2.0/saml-bindings-2.0-os.pdf - Page 18

I tried to find in Elastic or in ADFS some options to change the way URL are encoded but didn't find any.
Is there a way to do that in Elastic ?

Also, if the possible causes I thought of are wrong, can you please guide me and tell me where to look next ?

If you want some more information on the issue, please tell me.

Thank you for your time,
jptp

Which signature are you looking at?
This validation is using the Signature from the URL parameter. If the signature there is longer than 32 characters, can you provide the URL so we can work out why we only parsed a subset of it.

Thanks for your reply.

By the way, I am using Elastic v7.1.1, installed on a Debian 9 machine.

I replayed the logout scenario and this is what comes up in the logs :

org.elasticsearch.ElasticsearchSecurityException: SAML Signature [WNlkstu2IbQ348LYvjWyDg5sfQK9xf+7] could not be validated against [MIIC0DCCAbigAwIBAgIQIlVuniwQ8L1F3pYttLc4MjANBgkqhkiG9w0BAQsFADAk...,MIIDMTCCAhmgAwIBAgIIP3b3uW+4RU8wDQYJKoZIhvcNAQELBQAwHjEcMBoGA1UE...]

The two certificates listed match the ones I configured on the ADFS side as encryption and signing certificates.

Using Saml-tracer for Firefox, I retrieved the two SAML messages exchanged between ADFS and Elastic :

  • First message

https://iam-elastic.iam.lab:5601/logout?SAMLRequest=lVLda8MgEP9Xgu9Wm5oPJQ2U9SXQtbCOPeylGGM2IdHOM6N%2f%2fpKGMjZGYW%2fHnb8v7wqQfXcWO%2ffmhvCkPwYNIaq2a3SirGE1lQ2O45piRrnCUi4znCvKV5xpyhOJohftwTi7RvGCoqgCGHRlIUgbxhZdckwzvMyeaSbiXCTxImPsFUXbUcVYGa7I9xDOIAgxsse6k%2bNELcZ60claJCldku5qDkUPzoKeiAdvhZNgQFjZaxBBiePmcSdGD0LNj8Rg4ayVaY1uULR34WAPftMG7X%2f7Wq2%2bfV36zoK4fsl9lbN3wSnXobK4ZvYz9D5IAmg%2fZUbllHmM3MItKZFNC2ScfxqlgQQ%2fQCjIzF0W%2b5Gr2v5TY9P0xhoIXgbnCzJzlMW88KOGaW%2bVbfSlPGVNW%2bdtynCac4ZZSznmMeOYrbIkSeu0VrouyB%2fIW%2fPH%2fZRf&Signature=WNlkstu2IbQ348LYvjWyDg5sfQK9xf%2b7MS6yW9TayJGNoYg64Cd9xumyl5G48w93%2b5R94ew6vo2heGGistlwrFJQ5yz3OoBtRT0LbI7RvUKlV5ZOAeri5zNOt7qeYubDaIHGnIrVIcXPJ1I%2boN4JZnV5n%2ba9l9cABj3BaqzLa3oWjLx6Tx0RobLPilmc8B%2fx6kPVsNWtGGtAyRIWcHRWVXJbb8DmuY%2fFsr4LFzHqxkBN5q%2bhk3KMxIkduxB5MhK9JiUaAqj%2fOT0brEVIWsP0oNs6HjvFhiHKIDJ%2bjKIOiWQDa3yLZl90Hg0qZAin5Uuaoli0b%2bnZMpXe36BIu5MnGA%3d%3d&SigAlg=http%3a%2f%2fwww.w3.org%2f2001%2f04%2fxmldsig-more%23rsa-sha256

So the parameters are :

SAMLRequest : lVLda8MgEP9Xgu9Wm5oPJQ2U9SXQtbCOPeylGGM2IdHOM6N//pKGMjZGYW/Hnb8v7wqQfXcWO/fmhvCkPwYNIaq2a3SirGE1lQ2O45piRrnCUi4znCvKV5xpyhOJohftwTi7RvGCoqgCGHRlIUgbxhZdckwzvMyeaSbiXCTxImPsFUXbUcVYGa7I9xDOIAgxsse6k+NELcZ60claJCldku5qDkUPzoKeiAdvhZNgQFjZaxBBiePmcSdGD0LNj8Rg4ayVaY1uULR34WAPftMG7X/7Wq2+fV36zoK4fsl9lbN3wSnXobK4ZvYz9D5IAmg/ZUbllHmM3MItKZFNC2ScfxqlgQQ/QCjIzF0W+5Gr2v5TY9P0xhoIXgbnCzJzlMW88KOGaW+VbfSlPGVNW+dtynCac4ZZSznmMeOYrbIkSeu0VrouyB/IW/PH/ZRf

Signature : WNlkstu2IbQ348LYvjWyDg5sfQK9xf+7MS6yW9TayJGNoYg64Cd9xumyl5G48w93+5R94ew6vo2heGGistlwrFJQ5yz3OoBtRT0LbI7RvUKlV5ZOAeri5zNOt7qeYubDaIHGnIrVIcXPJ1I+oN4JZnV5n+a9l9cABj3BaqzLa3oWjLx6Tx0RobLPilmc8B/x6kPVsNWtGGtAyRIWcHRWVXJbb8DmuY/Fsr4LFzHqxkBN5q+hk3KMxIkduxB5MhK9JiUaAqj/OT0brEVIWsP0oNs6HjvFhiHKIDJ+jKIOiWQDa3yLZl90Hg0qZAin5Uuaoli0b+nZMpXe36BIu5MnGA==

SigAlg : http://www.w3.org/2001/04/xmldsig-more#rsa-sha256

This SAML request gets a 200 OK status code.

  • Second message

https://iam-elastic.iam.lab:5601/api/security/v1/logout?SAMLRequest=lVLda8MgEP9Xgu9Wm5oPJQ2U9SXQtbCOPeylGGM2IdHOM6N%2F%2FpKGMjZGYW%2FHnb8v7wqQfXcWO%2FfmhvCkPwYNIaq2a3SirGE1lQ2O45piRrnCUi4znCvKV5xpyhOJohftwTi7RvGCoqgCGHRlIUgbxhZdckwzvMyeaSbiXCTxImPsFUXbUcVYGa7I9xDOIAgxsse6k%2BNELcZ60claJCldku5qDkUPzoKeiAdvhZNgQFjZaxBBiePmcSdGD0LNj8Rg4ayVaY1uULR34WAPftMG7X%2F7Wq2%2BfV36zoK4fsl9lbN3wSnXobK4ZvYz9D5IAmg%2FZUbllHmM3MItKZFNC2ScfxqlgQQ%2FQCjIzF0W%2B5Gr2v5TY9P0xhoIXgbnCzJzlMW88KOGaW%2BVbfSlPGVNW%2BdtynCac4ZZSznmMeOYrbIkSeu0VrouyB%2FIW%2FPH%2FZRf&Signature=WNlkstu2IbQ348LYvjWyDg5sfQK9xf%2B7MS6yW9TayJGNoYg64Cd9xumyl5G48w93%2B5R94ew6vo2heGGistlwrFJQ5yz3OoBtRT0LbI7RvUKlV5ZOAeri5zNOt7qeYubDaIHGnIrVIcXPJ1I%2BoN4JZnV5n%2Ba9l9cABj3BaqzLa3oWjLx6Tx0RobLPilmc8B%2Fx6kPVsNWtGGtAyRIWcHRWVXJbb8DmuY%2FFsr4LFzHqxkBN5q%2Bhk3KMxIkduxB5MhK9JiUaAqj%2FOT0brEVIWsP0oNs6HjvFhiHKIDJ%2BjKIOiWQDa3yLZl90Hg0qZAin5Uuaoli0b%2BnZMpXe36BIu5MnGA%3D%3D&SigAlg=http%3A%2F%2Fwww.w3.org%2F2001%2F04%2Fxmldsig-more%23rsa-sha256

The parameters are the same.

This SAML request gets a 500 Internal Server Error status code.

I'm not really sure why there is two HTTP GET methods with identical SAML requests, might be related to the HTTP-Redirect binding.

I can post the whole HTTP request and respose if needed.

Thanks, I'm not sure what's going on, but you've given me enough info to go and see if I can reproduce it. We've had a bug regarding decoding of + in URLs, so it's possible that's the problem here, but it seems strange given that we are successfully decoding the + but then truncating a couple of characters later.

Unfortunately I don't have a work around for you right now, but I'll see if I can diagnose it and come back with more info.

Hello,

Do you have any updates concerning this issue ?