SAML SSO with WSO2 and Kibana

subhomoy.mukherjee · October 15, 2018, 11:26am

Hello,
I am new to Kibana and trying to work with WSO2 IS and Kibana for SSO. I want to embed Kibana dashboard in to my application (say appx) and using SAML SSO I want to achieve the following:

Login to appx
Goto dashboard option and able to view Kibana dashboard without any further logins. (i.e. SSO)

As per my understanding, the structure looks like: appx -> WSO2 <- Kibana

Now, when I try to access dashboard after logging in to appx, I get navigated to Kibana log on screen but the SSO does not happen. I am pretty sure that I am missing few steps here.
This is what I have done so far:

elasticsearch.yml:

#----------xpack-------
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
xpack.security.authc.token.enabled: true

#----------- SAML Realm -------------
xpack.security.authc.realms.saml1:
type: saml
order: 2
idp.metadata.path: saml/idp-metadata.xml
idp.entity_id: "https://wso2server:9443"
sp.entity_id: "http://localhost:5601/app/kibana"
sp.acs: "http://localhost:5601/app/kibana/api/security/v1/saml"
sp.logout: "http://localhost:5601/logout"
attributes.principal: "http://wso2.org/claims/username"
attributes.groups: "http://wso2.org/claims/country" (***this is an error in our application, we use the country for storing groups)
nameid_format: transient
#----------- Native Realm -------------
xpack.security.authc.realms.native1:
type: native
order: 0

kibana.yml:
xpack.security.audit.enabled: true
xpack.security.authProviders: [saml, basic]
server.xsrf.whitelist: [/api/security/v1/saml]

WSO2 IS:
Service Provider:

appx
Kibana:

Issuer: http://localhost:5601/app/kibana
ASC: http://localhost:5601/app/kibana/api/security/v1/saml
NameID: urn:oasis:names:tc:SAML:1.1:nameid-format:transient
Single Logout: http://localhost:5601/logout

Any help to solve this would be really appreciated. Kindly let me know if any other details are required. Thanks in advance.

ikakavas · October 15, 2018, 1:22pm

Hi there,

Can you clarify a few things first:

Does your appx application support SAML for SSO ?
Is WSO2 IS your SAML Identity Provider for your appx?

Can you clarify what does the

structure looks like: appx -> WSO2 <- Kibana

represent in your case? How do you want this to work?

get navigated to Kibana log on screen but the SSO does not happen.

What does happen, instead ?

Also,

. I want to embed Kibana dashboard in to my application

How do you plan to do this ? With an iframe?

Please also take a look at how the SSO flow for SAML2 looks like to get an overview.

We can get back to you with suggestions once you share more details

subhomoy.mukherjee · October 15, 2018, 1:43pm

Hi,
Thank you for your reply.

Yes, appx supports SAML for SSO, currently configured accordingly
Yes, WSO2 is the SAML IdP for appx

I mean to say that WSO2 is the identity provider for appx. In appx we have a dashboard view where we want to embed Kibana using iframe. So, when we login to appx using SSO, we should be able to login to Kibana dashboard as well with the single sign-on.
For example:

"user1" is a User created in WSO2 Identity Server.
user1 has the access to appx.
user1 logs in to appx, should be able to view KIbana dashboard as well.

Currently, when user1 logs in to appx and navigates to dashboard, the Kibana Log on screen is displayed, the default screen http://localhost:5601/login#?_g=()

Yes, that is correct. We are trying to implement with iframe.

I hope these details help. Please let me know if any other details are required.

ikakavas · October 15, 2018, 3:23pm

Ok, now I have a better idea of what you want to achieve. Two things I want to touch upon here.

1. Solution architecture

Step 3 above is not as simple as it sounds. What would basically happen is all the following:

User will try to access appx which will redirect them to the SAML Identity Provider ( WSO2 Identity Server )
User will authenticate with WSO2 IS by means of entering credentials in form and
a. A session cookie ( for WSO2 IS ) will be set in the user's browser
b. The user's browser will be redirected back to appx with a SAML Response
Appx will consume the SAML Response from WSO2 IS and after validating will create a session for the user in the appx's context ( setting again a appx specific session cookie ) .

Now when the user will try to access the page where Kibana is embedded

The browser will try to access kibana which will redirect them to the SAML Identity Provider ( WSO2 Identity Server ) . Kibana doesn't know that the user has already logged in to the Identity Provider. Kibana is a different Service Provider.
User will be redirected to the WSO2 IS within the iframe but this request will carry the existing cookies of the user (i.e. the cookie set in 2a above ) so that WSO2 IS can determine that the user is already authenticated and so it will not prompt for authentication but instead redirect the user back to Kibana with a SAML Response.
Kibana (and Elasticsearch) will handle the SAML Response as documented here and will set a Kibana specific session cookie for the user. Note that this cookie is unrelated to the appx cookie set in 3 above

A few potential issues with this:

You need to configure WSO2 IS user interface to be accessible from an iframe ( setting x-grame-option response header to an appropriate value ) otherwise step 5 above might fail.
You need to configure WSO2 IS for an appropriately large session duration so that when users open the dashboard page in appx, their original session with WSO2 IS is still valid. Otherwise they will be prompted for login within the iframe and that is not optimal UX
No matter how you configure things, keep in mind that the appx sessions and Kibana sessions are not connected somehow so your UI/UX should account for the fact that one of the two might expire before the other. i.e. it doesn't matter if you set your kibana sessions to expire after 3 hours if your appx sessions will expire after 10 minutes, your users will be prompted for authentication in appx even if they still have a valid session for kibana.

2. SAML Configuration

All the above will come in play after your correctly configure SAML for Kibana, as it now looks like it's not properly setup.

This means kibana falls back to using the basic authProvider because something goes wrong with handling the incoming SAML Response or because it can't even generate the SAML Request to WSO IS. You can find out what goes wrong by checking the logs from elasticsearch, we would have printed appropriate [ERROR] messages that can guide us through to resolution.

You can also enable more detailed logging for SAML by setting the appropriate level i.e.

PUT /_cluster/settings
{
  "transient": {
    "logger.org.elasticsearch.xpack.security.authc.saml": "trace"
  }
}

Hope this helps

subhomoy.mukherjee · October 16, 2018, 7:21am

Hello,
Thanks for your response. It did clear a lot of things for me.

Coming to the SAML configuration, I removed the basic authProvider and now I am getting the following on the screen:
{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred"}

The log says the following:
log [07:11:03.984] [info][authentication] Authentication attempt failed: [security_exception] Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=http://localhost:5601/api/security/v1/saml}]
error [07:11:03.959] [security_exception] Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=http://localhost:5601/api/security/v1/saml}] :: {"path":"/_xpack/security/saml/prepare","query":{},"body":"{"acs":"http://localhost:5601/api/security/v1/saml\"}","statusCode":500,"response":"{\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=http://localhost:5601/api/security/v1/saml}]"}],"type":"security_exception","reason":"Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=http://localhost:5601/api/security/v1/saml}]"},"status":500}"}
at respond (/home/smukherjee/Downloads/Kib/kibana-6.4.1-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:307:15)
at checkRespForFailure (/home/smukherjee/Downloads/Kib/kibana-6.4.1-linux-x86_64/node_modules/elasticsearch/src/lib/transport.js:266:7)
at HttpConnector. (/home/smukherjee/Downloads/Kib/kibana-6.4.1-linux-x86_64/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)
at IncomingMessage.bound (/home/smukherjee/Downloads/Kib/kibana-6.4.1-linux-x86_64/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)
at emitNone (events.js:111:20)
at IncomingMessage.emit (events.js:208:7)
at endReadableNT (_stream_readable.js:1064:12)
at _combinedTickCallback (internal/process/next_tick.js:138:11)
at process._tickDomainCallback (internal/process/next_tick.js:218:9)

ikakavas · October 16, 2018, 7:54am

You don't have to remove the basic authProvider, the log error would be caught eitherway.

The problem is outlined above. You have configured your Identity Provider and Elasticsearch with the Assertion Consumer Service URL of your Service Provider being http://localhost:5601/app/kibana/api/security/v1/saml

ASC: http://localhost:5601/app/kibana/api/security/v1/saml

sp.acs: "http://localhost:5601/app/kibana/api/security/v1/saml"

However, Kibana attempts to build the Assertion Consumer Service URL from what it knows about itself and from the request it receives and then passes it to Elasticsearch and it doesn't match as it looks like it's accessed/configured as http://localhost:5601/. ACS URL is matched as a string and doesn't involve any URL c14n so it needs to match exactly.

Can you tell us what the src attribute of your <iframe> is and your relevant kibana config ( server.name etc ) . Also, have you set any of the xpack.security.public config params ?

subhomoy.mukherjee · October 16, 2018, 8:17am

Hello,

I was not getting the error log without removing the basic authProvider, not sure why.

The src attribute is as follows:
<iframe src="http://localhost:5601/goto/76e72e0a90b67ca16ef7742f1c6d5cf1?embed=true" height="600" width="800"></iframe>

The Kibana config is as follows:
server.port: 5601
server.host: "localhost"
elasticsearch.username: "elastic"
elasticsearch.password: "elastic"
xpack.security.audit.enabled: true
xpack.security.authProviders: [saml, basic]
server.xsrf.whitelist: [/api/security/v1/saml]

No, there is no such config params.

ikakavas · October 16, 2018, 8:48am

You were not getting the error in the screen because kibana would fallback to basic, but the log messages would be there.

The easiest solution is to change your ACS in WSO2 IS and Elasticseach SAML Realm to be

ASC: http://localhost:5601/api/security/v1/saml

sp.acs: "http://localhost:5601/api/security/v1/saml"

---------------------------------------------------------------------

Totally unrelated to our discussion here but seeing this:

elasticsearch.username: "elastic"
elasticsearch.password: "elastic"

You should not use the elastic superuser for Kibana authenticating to Elasticsearch. There is a builtin kibana user that you should use.

subhomoy.mukherjee · October 16, 2018, 9:05am

ikakavas:

The easiest solution is to change your ACS in WSO2 IS and Elasticseach SAML Realm to be
ASC: http://localhost:5601/api/security/v1/saml 
sp.acs: "http://localhost:5601/api/security/v1/saml"

Did this, but now I get the following: "localhost refused to connect."
Could you please tell me which log file can I check to get a more detailed view about this?

ikakavas:

Totally unrelated to our discussion here but seeing this:
elasticsearch.username: "elastic"
elasticsearch.password: "elastic"
You should not use the elastic superuser for Kibana authenticating to Elasticsearch. There is a builtin kibana user that you should use.

Thanks for the suggestion, will use the 'Kibana' user.

ikakavas · October 16, 2018, 9:23am

Where do you get this ?

The elasticsearch log should have all information needed , you have already shared parts of it here. Look above where I have mentioned how you can enable more detailed logging . Please share some of it so that we can see what might be wrong

subhomoy.mukherjee · October 16, 2018, 9:39am

This is after clicking the dashboard where Kibana is embedded in appx using the src attribute. It just displays on screen.

There is no update in the log file for this scenario.

Also, I tried http://localhost:5601/ directly from the browser and now this is redirected to:

https://localhost:9443/samlsso?SAMLRequest=...

Note: The WSO2 IS is hosted on wso2server:9443

ikakavas · October 16, 2018, 11:12am

This is what is supposed to be happening. We ( ELK Stack - acting as a SAML Service Provider ) redirect you to the Identity Provider with a SAML Authentication Request. This is according to the SAML2 Web Browser SSO Profile.

My guess is that you are also redirected to WSO2 IS in the <iframe> but the Identity Provider user interface is not loading. You can validate this by looking at i.e. the network requests in the developer console in your browser. This is now becoming an "Load WSO2 IS in an iframe" problem and I'm afraid we won't be able to help you with this specific issue.

subhomoy.mukherjee · October 16, 2018, 12:05pm

Looking into the dev console, the request remains the same for the iframe:
https://localhost:9443/samlsso?SAMLRequest=..

But here I am not sure about one thing, that is why it redirects to localhost:9443 whereas the WSO2 IS is hosted in wso2server:9443 - Is this working as expected?

It would be really helpful if you can confirm on the below WSO2 IS configurations for the Kibana SP:

SAML2 Web SSO Configuration:
Issuer: http://localhost:5601/
ASC: http://localhost:5601/api/security/v1/saml
NameID format: urn:oasis:names:tc:SAML:2.0:nameid-format:transient
Enable Response Signing - Marked
Enable Single Logout -
SLO Response URL: http://localhost:5601/logout
SLO Request URL: http://localhost:5601/logout
Enable Attribute Profile - Yes
Include Attributes in the Response Always - Yes
Enable IdP Initiated SSO - Yes
Enable IdP Initiated SLO - Yes

ikakavas · October 16, 2018, 12:52pm

It redirects to localhost:9443 because you have configured it to redirect to localhost:9443 so yes, in a sense it is working as expected, but not correctly. Elasticsearch figures this ( it's called the SingleSignOnService URL in SAML2 ) by looking at the metadata that your SAML Identity Provider has created and that you have configured in

idp.metadata.path: saml/idp-metadata.xml

in your SAML Realm. Given the behavior you experience, I expect that you will find a line like the following in this file

 <md:SingleSignOnService Binding="urn:oasis:names:tc:SAML:2.0:bindings:HTTP-Redirect" Location="https://localhost:9443/samlsso"/>

This is wrong and should be fixed (since as you say WSO2 IS is not listening on 9443 on localhost).

I'm really sorry but I have no idea about WSO2 IS. You should look at its configuration or ask at an appropriate product discussion forum.

What does stand out is

Issuer: http://localhost:5601/

I would expect that to be a URL pointing to WSO2 IS as the Identity Provider is usually an Issuer (of a SAML Response or a SAML Assertion for instance) and it could be that your WSO2 IS instance is using this configuration to populate its metadata and this is why you get a wrong value in there.
But then again this becomes too WSO2 IS specific and I'm just guessing here. There are more appropriate forums to get help with WSO2 IS

subhomoy.mukherjee · October 16, 2018, 1:39pm

Thanks for pointing this out, I changed the SingleSignOnService URL to the WSO2 IS server and then I was getting the error for "x-frame-options" set to "Deny" in the dev console after clicking the dashboard.

Tried changing the x-frame to the WSO2 IS but getting an internal error on screen and the below in the logs:
TypeError: Header name must be a valid HTTP Token ["x-frame-options""]

Could you please advise?

ikakavas · October 16, 2018, 1:46pm

Yes, I had raised this as something you'd need to do already a few posts back:

Again, this is entirely out of scope for the purpose of this forum. This is an issue of how to load WSO2 IS in an <iframe> and has nothing to do with SAML or the ELK Stack. It's not that I don't want to help you but I don't know how as I have no experience whatsoever with WSO2 IS. I'd try to look up how to set or overwrite the x-frame-options header in WSO2 IS. The error message you shared indicates you tried that and something went wrong but I can't help more than that. Try here: Issues · wso2/product-is · GitHub maybe ?

subhomoy.mukherjee · October 16, 2018, 1:58pm

This is fine, I will have a look on the forum suggested by you. Thank you for this.

I tried another thing,
I set x-frame to deny once again and clicked the Request URL from the dev console i.e.

Refused to display 'https://wso2server:9443/samlsso?SAMLRequest=...' in a frame because it set 'X-Frame-Options' to 'deny'.

Due to this, I get the below in the logs:

[2018-10-16T15:50:39,652][WARN ][o.e.x.s.a.s.SamlAuthenticator] [hQ9aV9b] The XML Signature of this SAML message cannot be validated. Please verify that the saml realm uses the correct SAMLmetadata file/URL for this Identity Provider
[2018-10-16T15:50:39,655][WARN ][o.e.x.s.a.AuthenticationService] [hQ9aV9b] Authentication to realm saml1 failed - Provided SAML response is not valid for realm saml/saml1 (Caused by ElasticsearchSecurityException[SAML Signature [zmMT/0xpQWinHr+qhdmDMc0rGiw=
Sui...] could not be validated against []])

Is this message due to any configuration issue?

ikakavas · October 16, 2018, 2:07pm

Probably yes. It says it cannot verify that signature of the SAML Response. The SAML Response is signed with the private key of WSO2 IS and the signature is verified in Elasticsearch with the respective public key. The public key is contained in the SAML Metadata XML File that you created in WSO2 IS and you have configured in Elasticsearch. A number of things can go wrong:

Your metadata file does not contain a signing key
Your metadata file contains a corrupted signing key
Your metadata file contains the wrong signing key

Given the error message is could not be validated against []]) , the first case seem more probable. You need to double check that the configuration of WSO2 is correct and that the metadata is correct. You can always enable trace log in Elasticserach ( I have shared how above ) and see if there are additional messages that help us troubleshoot this.

subhomoy.mukherjee · October 17, 2018, 10:40am

I have updated the WSO2 configuration. Now I get the below info in the logs:

[info][audit][saved_objects_authorization_failure][security] sec_test unauthorized to get config, missing action:saved_objects/config/get
[info][audit][saved_objects_authorization_failure][security] sec_test unauthorized to get url, missing action:saved_objects/url/get
error  [10:32:41.167]  Error: Unable to get url, missing action:saved_objects/url/get
        at SecureSavedObjectsClient._execute

Note: "sec_test" is the user created in WSO2 IS.

ikakavas · October 17, 2018, 10:54am

You need to make sure your user has the necessary permissions to view the dashboard. This is managed by mapping the user to the appropriate roles in Elasticsearch.