X-pack - SAML issue - ELK 6.4

Elastic Stack Elasticsearch
1

Hello
I'm testing the ELK 6.4 version on stage env ( Ubuntu 16.04.5 LTS ) , i have 1LS with redis , 2 ES as data nodes and 1 Kib with ES as master. SSL setup went ok but but further I need to integrate it with SAML on Azure.

Kibana:
xpack.security.authProviders: [saml, basic]
server.xsrf.whitelist: [/api/security/v1/saml]

ES:

xpack.security.authc.realms.native1:
type: native
order: 0

xpack.security.authc.realms.saml1:
type: saml
order: 1
idp.metadata.path: elasticsearch.xml
idp.entity_id: "https://mydomain/b0384619-3635-481f-a15e-352939eed333/"
sp.entity_id: "https://mydomain:5601/"
sp.acs: "https://mydomain:5601/api/security/v1/saml"
sp.logout: "https://mydomain:5601/logout"
attributes.principal: "nameid:persistent"
attributes.groups: "roles"

In ES logs i have the following:

[2018-10-19T23:16:20,179][INFO ][o.o.c.c.InitializationService] Initializing OpenSAML using the Java Services API
[2018-10-19T23:16:21,096][INFO ][o.o.x.a.AlgorithmRegistry] Algorithm failed runtime support check, will not be usable: http://www.w3.org/2001/04/xmlenc#ripemd160
[2018-10-19T23:16:21,101][INFO ][o.o.x.a.AlgorithmRegistry] Algorithm failed runtime support check, will not be usable: http://www.w3.org/2001/04/xmldsig-more#hmac-ripemd160
[2018-10-19T23:16:21,109][INFO ][o.o.x.a.AlgorithmRegistry] Algorithm failed runtime support check, will not be usable: http://www.w3.org/2001/04/xmldsig-more#rsa-ripemd160
[2018-10-19T23:16:21,522][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] Metadata Resolver FilesystemMetadataResolver saml1: New metadata successfully loaded for '/etc/elasticsearch/elasticsearch.xml'
[2018-10-19T23:16:21,530][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] Metadata Resolver FilesystemMetadataResolver saml1: Next refresh cycle for metadata provider '/etc/elasticsearch/elasticsearch.xml' will occur on '2018-10-20T23:16:21.522Z' ('2018-10-20T23:16:21.522Z' local time)
[2018-10-19T23:16:21,573][INFO ][o.e.x.s.a.s.FileRolesStore] [domain] parsed [0] roles from file [/etc/elasticsearch/roles.yml]

the only strange this I see in logs is :
[WARN ][r.suppressed ] path: /_xpack/security/saml/prepare, params: {}
org.elasticsearch.ElasticsearchSecurityException: Cannot find any matching realm for [SamlPrepareAuthenticationRequest{realmName=null, assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml}]

but the GET response for _xpack/security/role_mapping is :
{
"saml-kibana": {
"enabled": true,
"roles": [
"kibana_user"
],
"rules": {
"field": {
"realm.name": "saml1"
}
},
"metadata": {}
},
"saml1": {
"enabled": true,
"roles": [
"kibana_user"
],
"rules": {
"field": {
"realm.name": "saml1"
}
},
"metadata": {}
}
}

.... and here I'm stuck . Please let me know if I should provide more logs and what am I missing :frowning:

I have the "xpack.security.authc.realms.saml1" configured just on the ES master node (should it be also on data nodes? ) Also the xml file is just on the master ES node.

Cheers!!!

2

This is the key error you need to work through.

It is possible to have multiple SAML realms in 1 Elasticsearch cluster (e.g. to support 2 separate Kibana deployments).
Kibana indicates which SAML realm is wishes to use by passing in the ACS URL of that Kibana instance.
In your case

  • you've told Kibana that it's on host 0.0.0.0 (per assertionConsumerServiceURL=https://0.0.0.0:5601/api/security/v1/saml)
  • but you've told Elasticsearch that Kibana is on mydomain (per sp.acs: "https://mydomain:5601/api/security/v1/saml")

You've got 3 basic options here:

  1. change the ES config to use 0.0.0.0 (this is probably not the right fix).
  2. tell Kibana to listen on a specific host/ip. See server.host in the Kibana settings
  3. leave Kibana listening on 0.0.0.0, but tell it that it's public URL (what is actually used in the browser) is "mydomain". See xpack.security.public.hostname in the SAML setup guide
3

Hi Tim,

I changed the Kibana to listen on ip (private ip) and I don't receive that error log anymore but:

ES logs:

[o.o.c.c.InitializationService] Initializing OpenSAML using the Java Services API
[2018-10-24T11:17:31,865][INFO ][o.o.x.a.AlgorithmRegistry] Algorithm failed runtime support check, will not be usable: http://www.w3.org/2001/04/xmlenc#ripemd160
[2018-10-24T11:17:31,869][INFO ][o.o.x.a.AlgorithmRegistry] Algorithm failed runtime support check, will not be usable: http://www.w3.org/2001/04/xmldsig-more#hmac-ripemd160
[2018-10-24T11:17:31,882][INFO ][o.o.x.a.AlgorithmRegistry] Algorithm failed runtime support check, will not be usable: http://www.w3.org/2001/04/xmldsig-more#rsa-ripemd160
[2018-10-24T11:17:32,564][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] Metadata Resolver FilesystemMetadataResolver saml1: New metadata successfully loaded for '/etc/elasticsearch/elasticsearch.xml'
[2018-10-24T11:17:32,574][INFO ][o.o.s.m.r.i.AbstractReloadingMetadataResolver] Metadata Resolver FilesystemMetadataResolver saml1: Next refresh cycle for metadata provider '/etc/elasticsearch/elasticsearch.xml' will occur on '2018-10-25T11:17:32.564Z' ('2018-10-25T11:17:32.564Z' local time)
[2018-10-24T11:17:32,617][INFO ][o.e.x.s.a.s.FileRolesStore] [esmaster.playngo.com] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2018-10-24T11:17:33,094][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/27722] [Main.cc@109] controller (64 bit): Version 6.4.2 (Build 660eefe6f2ea55) Copyright (c) 2018 Elasticsearch BV
[2018-10-24T11:17:33,442][DEBUG][o.e.a.ActionModule       ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-10-24T11:17:34,136][INFO ][o.e.d.DiscoveryModule    ] [hostname...] using discovery type [zen]
[2018-10-24T11:17:35,013][INFO ][o.e.n.Node               ] [hostname...] initialized
[2018-10-24T11:17:35,013][INFO ][o.e.n.Node               ] [hostname...] starting ...
[2018-10-24T11:17:35,175][INFO ][o.e.t.TransportService   ] [hostname] publish_address {172.25.7.211:9300}, bound_addresses {[::]:9300}

AND Kibana logs when a call is made:

Oct 24 11:27:10 stage-elk-ki-01 kibana[27342]: {"type":"log","@timestamp":"2018-10-24T11:27:10Z","tags":["debug","security","basic"],"pid":27342,"message":"Failed to authenticate request via header: [security_exception] unable to authenticate user [username@domain.com] for REST request [/_xpack/security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" & 1=\"Bearer realm=\\\"security\\\"\" } } }"}

Oct 24 11:21:01 stage-elk-ki-01 kibana[27342]: {"type":"response","@timestamp":"2018-10-24T11:21:01Z","tags":[],"pid":27342,"method":"post","statusCode":401,"req":{"url":"/api/security/v1/login","method":"post","headers":{"host":"hostname:5601","connection":"keep-alive","content-length":"67","accept":"application/json, text/plain, */*","origin":"https://hostname:5601","kbn-version":"6.4.2","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36","dnt":"1","content-type":"application/json;charset=UTF-8","referer":"https://hostname:5601/login","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en-US;q=0.9,en;q=0.8"},"remoteAddress":"local_ip","userAgent":"local_ip","referer":"https://hostname:5601/login"},"res":{"statusCode":401,"responseTime":64,"contentLength":9},"message":"POST /api/security/v1/login 401 64ms - 9.0B"}

So i get : 401 UNAUTHORIZED . Could be the reason that in order to enable SSL i created dns name inside the cluster and Azure doesn't know about it? For example kibanastage.domain.com point to kibana server and when the call is made from Azure it doesn't know about it?

Regards,
Robert

4

Hi Robert,

Please don't post unformatted logs as they're very hard to read.

Instead paste the text and format it with </> icon, and check the preview
window to make sure it's properly formatted before posting it. This makes it
more likely that your question will receive a useful answer.

It would be great if you could update your post to solve this.

Coming to your problem at hand:

  1. Your Elasticsearch log is from starting up and don't show anything wrong. The only SAML related part is that the metadata of your SAML IdP is loaded as expected.

  2. You say :

    AND Kibana logs when a call is made

    What call is made ?

It looks like you're trying to access https://hostname:5601/login. Going directly to /login allows you to use the basic authProvider of Kibana to authenticate (as you would also do without SAML). This is a suggested approach as it allows you to be able to login even if SAML is not working. You have enabled the basic authProvider because you have set

xpack.security.authProviders: [saml, basic]

in your kibana.yml.

You need to navigate to any other URL ( apart from /login) in order to start the SAML SSO flow.

If this still doesn't work, please share with us the relevant part of your elasticsearch.log as there will be more information there regarding what the problem might be.

5

Hi Ioannis,

xpack.security.authProviders: [saml, basic] is configured in kibana.yml , I generated another xml file in azure and imported in ES config.

When I try to login into Kibana now it does the redirect to Azure and ask for credentials but once I enter them I can see then following in ES logs:

 `Authentication to realm saml1 failed - SAML Attribute [nameid:persistent] for [attributes.principal] not found in [http://schemas.microsoft.com/identity/claims/tenantid=[b0384619-3635-481f-a15e-352939eed333], http://schemas.microsoft.com/identity/claims/objectidentifier=[0c17d0c1-5e7d-4e69-b693-1e46a6694792], http://schemas.microsoft.com/identity/claims/displayname=[Robert Galambos], http://schemas.microsoft.com/identity/claims/identityprovider=[https://sts.windows.net/b0384619-3635-481f-a15e-352939eed333/], http://schemas.microsoft.com/claims/authnmethodsreferences=[http://schemas.microsoft.com/ws/2008/06/identity/authenticationmethod/password], http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname=[Robert], http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname=[Galambos], http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress=[Robert.Galambos@domain.com], http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name=[Robert.Galambos@domain.com]]`

And kibana logs:

 kibana[30362]: {"type":"error","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","auth","session"],"pid":30362,"l                           evel":"error","error":{"message":"Unauthorized","name":"Error","stack":"Error: Unauthorized\n    at validate (/usr/share/kibana/node_modules/hapi-auth-cookie/                           lib/index.js:145:49)\n    at Object.authenticate (/usr/share/kibana/node_modules/hapi-auth-cookie/lib/index.js:210:13)\n    at module.exports.internals.Auth.i                           nternals.Auth.test (/usr/share/kibana/node_modules/hapi/lib/auth.js:96:22)\n    at Object.test (/usr/share/kibana/node_modules/hapi/lib/plugin.js:65:64)\n                               at resolve (/usr/share/kibana/node_modules/x-pack/plugins/security/server/lib/authentication/session.js:56:25)\n    at new Promise (<anonymous>)\n    at Sessi                           on.get (/usr/share/kibana/node_modules/x-pack/plugins/security/server/lib/authentication/session.js:55:12)\n    at Authenticator.authenticate (/usr/share/kiba                           na/node_modules/x-pack/plugins/security/server/lib/authentication/authenticator.js:137:49)\n    at Object.server.expose.request [as authenticate] (/usr/share/                           kibana/node_modules/x-pack/plugins/security/server/lib/authentication/authenticator.js:277:60)\n    at Object.authenticate (/usr/share/kibana/node_modules/x-p                               ack/plugins/security/server/lib/auth_redirect.js:30:60)\n    at request._protect.run (/usr/share/kibana/node_modules/hapi/lib/auth.js:324:34)\n    at module.e                           xports.internals.Protect.internals.Protect.run (/usr/share/kibana/node_modules/hapi/lib/protect.js:64:5)\n    at internals.Authenticator.execute (/usr/share/k                           ibana/node_modules/hapi/lib/auth.js:320:30)\n    at internals.Authenticator.authenticate (/usr/share/kibana/node_modules/hapi/lib/auth.js:306:21)\n    at modu                           le.exports.internals.Auth.internals.Auth._authenticate (/usr/share/kibana/node_modules/hapi/lib/auth.js:214:19)\n    at internals.Auth.authenticate (/usr/shar                           e/kibana/node_modules/hapi/lib/auth.js:202:17)\n    at each (/usr/share/kibana/node_modules/hapi/lib/request.js:384:16)\n    at iterate (/usr/share/kibana/nod                           e_modules/items/lib/index.js:36:13)\n
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"Try                           ing to authenticate via header."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"SAM                           L response payload is not found."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"Try                           ing to initiate SAML handshake."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"Red                           irecting to Identity Provider with SAML request."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"response","@timestamp":"2018-10-24T14:18:02Z","tags":[],"pid":30362,"method":"get","statusCode":302,"r                           eq":{"url":"/","method":"get","headers":{"host":"kibanastage.playngo.com:5601","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla                           /5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36","dnt":"1","accept":"text/html,application/xhtml+                           xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","accept-encoding":"gzip, deflate, br","accept-language":"en-GB,en-US;q=0.9,en;q=0.8"},"remoteAddres                           s":"10.100.13.51","userAgent":"10.100.13.51"},"res":{"statusCode":302,"responseTime":7,"contentLength":9},"message":"GET / 302 7ms - 9.0B"}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","legacy-platform-proxifier"],"pid":30362,"mes                           sage":"Request will be handled by proxy POST:/api/security/v1/saml."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"Try                           ing to authenticate user request to /api/security/v1/saml."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"Try                           ing to authenticate via header."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"Aut                           horization header is not presented."}
    Oct 24 14:18:02 stage-elk-ki-01 kibana[30362]: {"type":"log","@timestamp":"2018-10-24T14:18:02Z","tags":["debug","security","saml"],"pid":30362,"message":"Try                           ing to authenticate via state."}

Could be something wrong with xml file?
Or realm settings from ES?

Regards,
Robert

6

It's your configuration. The error is quite descriptive :

 Authentication to realm saml1 failed - 

SAML Attribute [nameid:persistent] for [attributes.principal] not found in 

[http://schemas.microsoft.com/identity/claims/tenantid=[b0384619-3635-481f-a15e-352939eed333],
 http://schemas.microsoft.com/identity/claims/objectidentifier=[0c17d0c1-5e7d-4e69-b693-1e46a6694792], 
http://schemas.microsoft.com/identity/claims/displayname=[Robert Galambos], 
http://schemas.microsoft.com/identity/claims/identityprovider=[https://sts.windows.net/b0384619-3635-481f-a15e-352939eed333/], 
http://schemas.microsoft.com/claims/authnmethodsreferences=[http://schemas.microsoft.com/ws/2008/06/identity/authenticationmethod/password],
 http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname=[Robert], 
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname=[Galambos], 
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress=[Robert.Galambos@domain.com],
 http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name=[Robert.Galambos@domain.com]]

You have configured your SAML realm to try and map the peristent NameID ( whcih is Subject of the SAML Assertion ) to the principal attribute.

attributes.principal: "nameid:persistent"

The SAML Response that Azure sends does not contain a persistent NameID so the mapping fails.
The principal attribute is mandatory so if this fails to be mapped, then the authentication fails.

Judging by what is available in the response ( the list that we print in the log and I have copied to my post ), there is a number of possible attributes that you can use for mappging to the principal.

For instance

attributes.principal: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name

You can read more about the attribute mapping and your options here

7

After changing

from
attributes.principal: "nameid:persistent"
to
attributes.principal: http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name
login works !!!

Thank you for your time !

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.