SAML Group Attributes with more than one group

Hi All,

I am trying to integrate Kibana running on Kuberenetes with on-prem ADFS with the below mapping.

SAML Config
SAML Config
saml:
saml1:
order: 1
ssl.verification_mode: none
idp.metadata.path: adfs.xml
idp.entity_id: "http://<ADFS_LINK>"
sp.entity_id: "https://<KIBANA_LINK>"
sp.acs: "https://<KIBANA_LINK>/api/security/v1/saml"
sp.logout: "https://<KIBANA_LINK>/logout"
attributes.principal: "nameid:persistent"
attributes.groups: "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"

{
"type": "server",
"timestamp": "2019-10-25T15:22:49,133+0000",
"level": "DEBUG",
"component": "o.e.x.s.a.s.SamlRealm",
"cluster.name": "elasticsearch",
"node.name": "elasticsearch-master-2",
"cluster.uuid": "DNY-11",
"node.id": "NP5gGFycRZ-",
"message": "Parsed token [SamlToken{}] to attributes [SamlAttributes(null)[null]{[nameid:persistent=[username@email.com], http://schemas.xmlsoap.org/claims/Group=[Domain Users, APP1_Group, APP2_Group, ELK_Group, APP4_Group]]}]"
}

I would like to do the SAML role Mapping for all the users in the group ELK_Group to have superuser privileges. However, the below role_mapping is not working.

PUT /_security/role_mapping/saml-superuser
{
"roles": [ "superuser" ],
"enabled": true,
"rules": { "all": [
{ "field": { "realm.name": "saml1" } },
{ "field": { "Group": "ELK_Group" } }
]
}
}

Can anyone help

Thanks
Ram

Hi @ramss,

I see a typo in your request for creating role-mapping.

Instead of Group the field name should be groups.

The user fields that are provided to the role mapping are derived from the SAML attributes as follows:

  • username : The principal attribute
  • dn : The dn attribute
  • groups : The groups attribute

Referenced from the docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/saml-role-mapping.html

Hope this helps.

Thanks and Regards,
Yogesh Gaikwad

Hi Yogesh,
Thanks for your reply. Changed and tested

{"statusCode":403,"error":"Forbidden","message":"Forbidden"}

I have changed that to groups, still there is no progress. Basically, it is not mapping properly as the return gives a list and mapping is matching with a string "ELK_Group", there should be a mechanism to search through the list provided
[SamlAttributes(null)[null]{[nameid:persistent=[[username@email.com](mailto:username@email.com)], http://schemas.xmlsoap.org/claims/Group=[Domain Users, APP1_Group, APP2_Group, ELK_Group, APP4_Group]]}]"
and then pick the ELK_Group. If he is part of any ELK group, then I should do the role mapping.

Came across role_templates:

POST /_security/role_mapping/adfs_role_owner
{
  "role_templates": [
    {
      "template": { "source": "{\"query\":{\"match\":{\"title\":\"{{query_string}}\"}}}"
                    "{{#tojson}}http:\/\/schemas.xmlsoap.org\/claims\/Group{{/tojson}}" }, 
      "format" : "json" 
    }
  ],
  "rules": { all: [ 
    { "field": { "realm.name" : "saml1" } },
    { "field": { "Group": "ELK_Group" } }
  ]
  },
  "enabled": true
}```

However, this is not solving as well. Any idea???

Thanks & Regards
Ram

Hi @ramss,

In your SAML realm configuration I see that the attribute name is different than the list of SAML attributes that are parsed from the token.
From elasticsearch.yml:
attributes.groups: "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"
and from the log message ""

"message": "Parsed token [SamlToken{}] to attributes [SamlAttributes(null)[null]{[nameid:persistent=[username@email.com], http://schemas.xmlsoap.org/claims/Group=[Domain Users, APP1_Group, APP2_Group, ELK_Group, APP4_Group]]}]"

I think we should be using the http://schemas.xmlsoap.org/claims/Group as the attributes.groups: setting in elasticsearch.yml.

This might be the reason for the role mapping failure, can you please try changing this and see how it goes.

Regards,
Yogesh Gaikwad

It looks like you have a misconfiguration here.
nameid:persistent=[[username@email.com](mailto:username@email.com)]
This means that you have configured ADFS to release a SAML attribute with the name nameid:persistent. That works, but it's always a bad idea.

In Elasticsearch nameid:persistent is a magic value that means Don't read the value from a SAML attribute, read it from the <NameID> (but only if the nameid is persistent).
Naming an attribute nameid:persistent is just asking for trouble.

1 Like

Hi Yogesh,

I tried changing the Group and received the below error.

{"type": "server", "timestamp": "2019-10-29T08:34:00,125+0000", "level": "WARN", "component": "o.e.x.s.a.AuthenticationService", "cluster.name":
"elasticsearch", "node.name": "elasticsearch-master-1", "cluster.uuid": "DNY-13ojQOKW5ex36jzTJw", "node.id": "Ngh6YsWIQQ2TfRZkkoeetA", "messag
e": "An error occurred while attempting to authenticate [] against realm [saml1]" ,
"stacktrace": ["org.elasticsearch.common.xcontent.XContentParseException: Roles template must generate a string or an array of strings, but foun
d [null]",
"at org.elasticsearch.xpack.core.security.authc.support.mapper.TemplateRoleName.convertJsonToList(TemplateRoleName.java:121) ~[?:?]",
"at org.elasticsearch.xpack.core.security.authc.support.mapper.TemplateRoleName.getRoleNames(TemplateRoleName.java:91) ~[?:?]",
"at org.elasticsearch.xpack.core.security.authc.support.mapper.ExpressionRoleMapping.lambda$getRoleNames$3(ExpressionRoleMapping.java:258) ~[?:?
]",

I am new to Elastic as well as SAML. sorry for very basic questions. Your help would be much appreciated

Thanks
Ram

You need to tell us exactly what you changed, so please share your updated configuration.

Also , you shouldn't need to user role templates for your use case. If I understand this correctly, you need all users that are members of the ELK_Group to get the superuser role, correct ?

Then, the following:

PUT /_security/role_mapping/saml-superuser
{
  "roles": [ "superuser" ],
  "enabled": true,
  "rules": { "all": [
      { "field": { "realm.name": "saml1" } },
      { "field": { "groups": "ELK_Group" } }
    ]
  }
}

should be enough.

So in summary, and as @Yogesh_Gaikwad already suggested above:

  1. Change

    attributes.groups: "http://schemas.microsoft.com/ws/2008/06/identity/claims/role"
    

    to

    attributes.groups: "http://schemas.xmlsoap.org/claims/Group"
    
  2. Create the role mapping, as I shared it above.

Your users will then be able to authenticate and the members of the ELK_Group will get the superuser role in elasticsearch

Hi Tim,

Thanks for your reply. I have added your changes and I think it is matching the values now. However, the TRACE/DEBUG error messages are not informing me a lot. Would you mind looking at it once please? I have also added changes which was mentioned by Yogesh.

ADFS Config:
=> issue(store = "Active Directory", types = ("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier",
"http://schemas.xmlsoap.org/claims/Group"), query =
";userPrincipalName,tokenGroups;{0}", param = c.Value);

SAML Config:
saml:
saml1:
order: 1
ssl.verification_mode: none
idp.metadata.path: adfs.xml
idp.entity_id: "http://ADFS_LINK"
sp.entity_id: "https://KIBANA_LINK"
sp.acs: "https://KIBANA_LINK/api/security/v1/saml"
sp.logout: "https://KIBANA_LINK/logout"
attributes.principal: "nameid:persistent"
attributes.groups: "http://schemas.xmlsoap.org/claims/Group"
nameid_format: urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified

Error Messages I am getting now:

[SamlToken{3c73616d--------------2049443d225f63306433383232352d393961622d343939302d396562392d63353230393735316634393622205665727369...}] to attributes [SamlAttributes(NameId(null)=username@email.com) {[_3f3a807d-----4810-84d0-1ecd5faa62b4]{[http://schemas.xmlsoap.org/claims/Group=[Domain Users, APP1_Group, APP2_Group, ELK_Group, APP4_Group]]}]"

And also the browser display a error messages, where the realm is switched back to basic
{"statusCode":401,"error":"Unauthorized","message":"[security_exception] unable to authenticate user [] for action [cluster:admin/xpack/security/saml/authenticate], with { header={ WWW-Authenticate={ 0="Bearer realm=\"security\"" & 1="ApiKey" & 2="Basic realm=\"security\" charset=\"UTF-8\"" } } }"}

Thanks & Regards
Ram

Thanks Ioannis, I have changed that configuration and I am receiving the below error
{"type": "server", "timestamp": "2019-10-29T11:54:48,095+0000", "level": "WARN", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-2", "cluster.uuid": "DNY-13o", "node.id": "NP5gGFycRZ-3rXew6H_GmA", "message": "Authentication to realm saml1 failed - SAML Attribute [nameid:persistent] for [xpack.security.authc.realms.saml.saml1.attributes.principal] not found in saml attributes[http://schemas.xmlsoap.org/claims/Group=[Domain Users, APP1_Group, APP2_Group, ELK_Group, APP4_Group]]}]"

Let's take a step back. Many of us chimed in to help and it might got confusing, apologies. The latest error you get now isn't related to the changes I suggested, but to the changes you did due to what Tim was suggesting, which unfortunately weren't enough to address his points.

It doesn't. What you have done now is configure ADFS to release a NameID without a format. This is again almost always wrong. You don't have to use a NameID, but if you do, you need to configure it correctly. We can't help you much with your ADFS configuration, but you need to

  • either figure out how to release a NameID with persistent format ( based on the value of some attribute ) and configure elasticsearch to map this to the principal property
  • or use any other attribute to map to the principal property. I.e. UPN

The possibilities for correct configuration are many and there isn't a better/simple/only one I can suggest. So I'll suggest one that works and feel free to alter this to your liking.

  1. Configure ADFS to release a persistent NameID. There's enough ADFS related documentation around this, i.e. see https://blogs.msdn.microsoft.com/card/2010/02/17/name-identifiers-in-saml-assertions/ and check the section with subtitle " Persistent name identifier". You need to select which value AD you would pass as the value of the NameID ( == what you should select for `Incoming claim type" in the screenshot ), UPN is a good candidate.

  2. Change nameid_format: urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified to nameid_format: urn:oasis:names:tc:SAML:2.0:nameid-format:persistent in your configuration.

  3. Leave everything else as is.

Hi
Thanks for the clear steps

The reason I made it nameid_format: urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified because the SAML Response was giving the invalidNameIDPolicy

{"type": "server", "timestamp": "2019-10-29T14:46:03,856+0000", "level": "WARN", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1", "cluster.uuid": "DNY-13ojQOKW5ex36jzTJw", "node.id": "Ngh6YsWIQQ2TfRZkkoeetA",  "message": "Authentication to realm saml1 failed - Provided SAML response is not valid for realm saml/saml1 (Caused by ElasticsearchSecurityException[SAML Response is not a 'success' response: The SAML IdP did not grant the request. It indicated that the Elastic Stack side sent something invalid (urn:oasis:names:tc:SAML:2.0:status:Requester). Specific status code which might indicate what the issue is: [urn:oasis:names:tc:SAML:2.0:status:InvalidNameIDPolicy]])"  }

{"type": "server", "timestamp": "2019-10-29T14:46:04,314+0000", "level": "TRACE", "component": "o.e.x.s.a.s.SamlRealm", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-1", "cluster.uuid": "DNY-13ojQOKW5ex36jzTJw", "node.id": "Ngh6YsWIQQ2TfRZkkoeetA",  "message": "Constructed SAML Authentication Request: <?xml version=\"1.0\" encoding=\"UTF-8\"?><saml2p:AuthnRequest xmlns:saml2p=\"urn:oasis:names:tc:SAML:2.0:protocol\" AssertionConsumerServiceURL=\"https://kibana/api/security/v1/saml\" Destination=\"https://adfs\" ID=\"_24ec5a5208c98653f89406dfd241d5e7def676a9\" IssueInstant=\"2019-10-29T14:46:04.312Z\" ProtocolBinding=\"urn:oasis:names:tc:SAML:2.0:bindings:HTTP-POST\" Version=\"2.0\">\n  <saml2:Issuer xmlns:saml2=\"urn:oasis:names:tc:SAML:2.0:assertion\">https://kibana</saml2:Issuer>\n  <saml2p:NameIDPolicy AllowCreate=\"false\" Format=\"urn:oasis:names:tc:SAML:2.0:nameid-format:persistent\"/>\n</saml2p:AuthnRequest>\n"  }

Does it still throw that error, even now after following the steps above when you have configure it to release a persistent Name ID ?

Yes. I have made the changes and those error messages are after the changes.

If you add it back, does it otherwise work with the instructions from the last post ?

Yes I have reverted the configuration and I get the previous error
{"type": "server", "timestamp": "2019-10-29T11:54:48,095+0000", "level": "WARN", "component": "o.e.x.s.a.AuthenticationService", "cluster.name": "elasticsearch", "node.name": "elasticsearch-master-2", "cluster.uuid": "DNY-13o", "node.id": "NP5gGFycRZ-3rXew6H_GmA", "message": "Authentication to realm saml1 failed - SAML Attribute [nameid:persistent] for [xpack.security.authc.realms.saml.saml1.attributes.principal] not found in saml attributes[http://schemas.xmlsoap.org/claims/Group=[Domain Users, APP1_Group, APP2_Group, ELK_Group, APP4_Group]]}]"

Did you do what I suggested on SAML Group Attributes with more than one group , especially in point 1. ? It doesnt look like you did that. Do only 1. and not 2.

Yes, I did the following changes at the AD level to transform

IssuanceTransformRules               : @RuleTemplate = "LdapClaims"
                                       @RuleName = "UPN"
                                       c:[Type == 
                                       "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", 
                                       Issuer == "AD AUTHORITY"]
                                        => issue(store = "Active Directory", types = 
                                       ("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier", 
                                       "http://schemas.xmlsoap.org/claims/Group"), query = 
                                       ";userPrincipalName,tokenGroups;{0}", param = c.Value);
                                       
                                       @RuleTemplate = "MapClaims"
                                       @RuleName = "Name ID Persistent"
                                       c:[Type == 
                                       "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier", Properti
                                       es["http://schemas.xmlsoap.org/ws/2005/05/identity/claimproperties/format"] == 
                                       "urn:oasis:names:tc:SAML:1.1:nameid-format:unspecified"]
                                        => issue(Type = 
                                       "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier", Issuer 
                                       = c.Issuer, OriginalIssuer = c.OriginalIssuer, Value = c.Value, ValueType = 
                                       c.ValueType, Properties["http://schemas.xmlsoap.org/ws/2005/05/identity/claimpro
                                       perties/format"] = "urn:oasis:names:tc:SAML:2.0:nameid-format:persistent");

I am not sure if this is a correct ADFS configuration but it looks like ADFS still doesn't send a persistent NameId.

Please see on https://www.elastic.co/guide/en/elasticsearch/reference/7.4/trb-security-saml.html how to enable trace logging ( towards the bottom of the page ) and share your elasticsearch logs from a failed authentication with us please.

Hi,

Apologies for the delay in response. Got into another production issue and got carried away. Managed to login to kibana with just user ID

PUT /security/role_mapping/saml-kibana
{
  "roles": [ "kibana_user" ],
  "enabled": true,
  "rules": {
    "field": { "realm.name": "saml1" }
  }
}

This works!!!

I am into role_mapping now. Came across role_templates --> https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-put-role-mapping.html

POST /_security/role_mapping/mapping5
{
  "role_templates": [
    {
      "template": { "source": "{{#tojson}}groups{{/tojson}}" }, 
      "format" : "json" 
    }
  ],
  "rules": {
    "field" : { "realm.name" : "saml1" }
  },
  "enabled": true
}

However, I am not sure how many groups will be added in future and ideally I should not be maintaining the groups if I am enabling SAML

I should pick and choose the user and assign him different role_mappings as elk_admin, elk_reader, elk_bus_user with relevant privileges.

User 1: ram-admin : Domain User, elk_admin, k8s_admin - role_mapping: elk_admin_mapping
User 2: ram-reader: Domain User, gcp_reader, elk_reader - role_mapping: elk_reader_mapping
...

Is this possible via role_templates to only choose the highlighted groups and assign the role mapping accordingly

Thanks & Regards
Ram

You don't assign role mappings to users. You create role mappings that assign roles to users. The role mappings contain logic that dictates which roles users will get depending on some of their attributes in AD.

It is possible but you don't need role templates for this, this is a very common case and we already handle this. Check our docs https://www.elastic.co/guide/en/elasticsearch/reference/7.4/saml-role-mapping.html

As an example, If you want the users that are in the AD Group elk_admin to get the Elasticsearch role ram-admin you need something like:

PUT /_security/role_mapping/saml-mapping-1
{
  "roles": [ "ram-admin" ],
  "enabled": true,
  "rules": { "all": [
        { "field": { "realm.name": "saml1" } },
        { "field": { "groups": "elk_admin" } }
  ] }
}