SOLVED:Issue in LDAP group authentication (shield):

Elastic Stack Elasticsearch
1

Shield v 2.3.2
If a add user in role_mapping file , user can authenticate against LDAP:

  • "uid=testuser,ou=People,dc=company,dc=com""

But if a try to add group (to which this user is a member) i get an authentication error:

Below are results from ldapsearch :
ldapsearch -D "cn=directory manager" -b "ou=People,dc=company,dc=com" -L -s sub -v "uid=testuser" isMemberOf
bin_ldapsearch: started Wed May 25 18:10:07 2016

ldap_init( localhost, 389 )
filter pattern: uid=testuser
returning: isMemberOf
filter is: (uid=testuser)
version: 1
isMemberOf: cn=Users,ou=group,dc=company,dc=com
isMemberOf: cn=SuperUsers,ou=group,dc=company,dc=com
isMemberOf: cn=SecurityUsers,ou=group,dc=company,dc=com

I have added below in role_mapping:

admin:

  • "cn=SuperUsers,ou=group,dc=company,dc=com"

Added below in elasticsearch.yml

shield:
authc:
realms:
esusers:
type: esusers
order: 0
ldap1:
type: ldap
order: 1
url: "ldaps://ldap.company.com:636"
user_dn_templates:
- "uid={0},ou=People,dc=company,dc=com"
group_search:
base_dn: "ou=group,dc=company,dc=com"
files:
role_mapping: "/path/to/role_mapping.yml"
unmapped_groups_as_roles: false

2

The most helpful way to debug these types of issues is to increase the logging. In the regular logging.yml file in the elasticsearch config directory, add the following in the logger section

shield.authc: TRACE

Now when you try to authenticate as this user, you will see the list of group DNs that were retrieved for the user in the logs. I'd first check to ensure that you are getting groups back. It may be that the type of groups in your ldap server requires a different filter.

3

Thanks Jay-

Added shield.authc: TRACE in logging.yml

Getting below error:

[2016-05-26 12:45:46,441][DEBUG][shield.authc.ldap ] [mm-dataH] authentication failed for user [testuser]
java.io.IOException: failed to connect to any LDAP servers
at org.elasticsearch.shield.authc.ldap.LdapSessionFactory.getSession(LdapSessionFactory.java:74)
at org.elasticsearch.shield.authc.ldap.support.SessionFactory.session(SessionFactory.java:98)
at org.elasticsearch.shield.authc.ldap.support.AbstractLdapRealm.doAuthenticate(AbstractLdapRealm.java:50)
at org.elasticsearch.shield.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:92)

Caused by: LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to connect to server ldap.company.com:636: java.io.IOException: Unable to verify an attempt to to establish a secure connection to 'ldap.company.com:636' because an unexpected error was encountered during validation processing: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated')

    at org.elasticsearch.shield.authc.ldap.LdapSessionFactory.getSession(LdapSessionFactory.java:72)
    ... 53 more

Caused by: java.io.IOException: Unable to verify an attempt to to establish a secure connection to 'ldap.company.com:636' because an unexpected error was encountered during validation processing: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
at com.unboundid.ldap.sdk.LDAPConnectionInternals.(LDAPConnectionInternals.java:158)
at com.unboundid.ldap.sdk.LDAPConnection.connect(LDAPConnection.java:859)

4

Did you import the certificate of your ldap server into the truststore or keystore? https://www.elastic.co/guide/en/shield/current/ldap-realm.html#_encrypting_communications_between_shield_and_ldap_with_ssl_tls

5

Yes i did. I wonder why this error is coming because "user level authentication is working fine".

This error comes when i add group in role_mapping file !!

6

Did you also add the group search at the same time? If so, it may be caused by a referral being followed that the certificate is not trusted for. In your ldap realm configuration, you can try adding follow_referrals: false

7

Yes, group_search was also added. Added follow_referrals but git the exact same error. Below is how config looks like now:

shield:
authc:
realms:
esusers:
type: esusers
order: 0
ldap1:
type: ldap
order: 1
url: "ldaps://ldap.company.com:636"
user_dn_templates:
- "uid={0},ou=People,dc=company,dc=com"
group_search:
base_dn: "ou=group,dc=company,dc=com"
files:
role_mapping: "/path/to/role_mapping.yml"
unmapped_groups_as_roles: false
follow_referrals: false

Also , in our LDAP we dont have user_group_attribute associated with member.

8

Do you know if the ldap servers are load-balanced? Or maybe the DNS entry points to multiple servers, which you can check with nslookup ldap.company.com

9

Right , same DNS entry is pointing to multiple IP's

10

Ok, does each IP have a different certificate? If so, that would explain the issue. After you restarted, Java resolved the DNS name to a different IP address.

You can check with `openssl s_client -showcerts -connect ip:port``` to each individual IP and see if the certificates are the same. Are the certificates self-signed or signed by the same certificate authority? If they are signed by a certificate authority and you can get that certificate; importing the certificate authority certificate should be enough.

11

Thanks for the tip Jay! I'l try this.

12

I have imported the ca cert and getting below error now: I think am pretty close , may be need some filter to get through.

[2016-05-27 09:58:16,352][DEBUG][rest.suppressed ] /_cluster/health Params: {pretty=true}
ElasticsearchSecurityException[action [cluster:monitor/health] is unauthorized for user [testuser]]
at org.elasticsearch.shield.support.Exceptions.authorizationError(Exceptions.java:45)
at org.elasticsearch.shield.authz.InternalAuthorizationService.denialException(InternalAuthorizationService.java:322)
at org.elasticsearch.shield.authz.InternalAuthorizationService.denial(InternalAuthorizationService.java:296)
at org.elasticsearch.shield.authz.InternalAuthorizationService.authorize(InternalAuthorizationService.java:148)
at org.elasticsearch.shield.action.ShieldActionFilter.apply(ShieldActionFilter.java:107)
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:170)
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:144)
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:85)
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:58)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:52)
at org.elasticsearch.rest.BaseRestHandler$HeadersAndContextCopyClient.doExecute(BaseRestHandler.java:83)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:359)
at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.execute(AbstractClient.java:853)
at org.elasticsearch.client.support.AbstractClient$ClusterAdmin.health(AbstractClient.java:873)
at org.elasticsearch.rest.action.admin.cluster.health.RestClusterHealthAction.handleRequest(RestClusterHealthAction.java:62)
at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:54)
at org.elasticsearch.rest.RestController.executeHandler(RestController.java:205)
at org.elasticsearch.rest.RestController$RestHandlerFilter.process(RestController.java:279)
at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:260)
at org.elasticsearch.shield.rest.ShieldRestFilter.process(ShieldRestFilter.java:79)
at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:263)
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:176)
at org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:128)
at org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:86)
at org.elasticsearch.http.netty.NettyHttpServerTransport.dispatchRequest(NettyHttpServerTransport.java:449)

13

Based on that. Authentication appears to be working but the role mapping still is having issues. Do you still have trace logging on? If so you should see some log lines about roles being mapped based on retrieved groups.

Do you know the schema of your LDAP system? By default the groupsearch looks for objects with an objectClass of group, groupOfNames, or groupOfUniqueNames, with the attributes member or memberOf that match the user's DN. I know that some LDAP systems use a posixGroup object class with a different attribute and this requires a custom filter in the realm settings.

14

Trace is still on , getting below error. It seems its not able to map any role to user. Schema name is rfc2307

[2016-05-27 10:41:31,547][DEBUG][shield.authc.ldap ] authenticated user [testuser], with roles [[]]

But when i add user - logs shows user has been mapped to admin role
[2016-05-27 10:55:54,364][DEBUG][shield.authc.support ] the roles [[admin]], are mapped from the user [ldap] for realm [uid=testuser,ou=people,dc=company,dc=com/ldap]

15

I think based on the schema you need to use a different filter for posixGroups. Try the following group_search settings in your realm configuration:

group_search:
  base_dn: "ou=group,dc=company,dc=com"
  filter: "(&(objectclass=posixGroup)(memberUID={0}))"
  user_attribute: "uid"

Alternatively, based on the ldapsearch in your initial post you could try the following realm configuration:

ldap1:
type: ldap
order: 1
url: "ldaps://ldap.company.com:636"
user_dn_templates:
  - "uid={0},ou=People,dc=company,dc=com"
user_group_attribute: "isMemberOf"

Also, did you see a log line that looked like the one below?

the roles [], are mapped from these [ldap] groups [...] for realm [ldap/ldap1]
16

On using first config , getting below error
[2016-05-27 11:10:22,148][DEBUG][shield.authc.ldap ] user not found in cache, proceeding with normal authentication [2016-05-27 11:10:22,208][DEBUG][shield.authc.ldap ] authentication failed for user [testuser] java.lang.NullPointerException at org.elasticsearch.shield.authc.ldap.SearchGroupsResolver.readUserAttribute(SearchGroupsResolver.java:83) at org.elasticsearch.shield.authc.ldap.SearchGroupsResolver.resolve(SearchGroupsResolver.java:63) at org.elasticsearch.shield.authc.ldap.support.LdapSession.groups(LdapSession.java:77) at org.elasticsearch.shield.authc.ldap.support.AbstractLdapRealm.createUser(AbstractLdapRealm.java:89) at org.elasticsearch.shield.authc.ldap.support.AbstractLdapRealm.doAuthenticate(AbstractLdapRealm.java:51) at org.elasticsearch.shield.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:92) at org.elasticsearch.shield.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:32) at org.elasticsearch.shield.authc.InternalAuthenticationService.authenticate(InternalAuthenticationService.java:363) at org.elasticsearch.shield.authc.InternalAuthenticationService.authenticate(InternalAuthenticationService.java:109) at org.elasticsearch.shield.rest.ShieldRestFilter.process(ShieldRestFilter.java:73) [2016-05-27 11:10:22,210][DEBUG][rest.suppressed ] /_cluster/health Params: {pretty=true} ElasticsearchSecurityException[unable to authenticate user [testuser] for REST request [/_cluster/health?pretty=true]] at org.elasticsearch.shield.support.Exceptions.authenticationError(Exceptions.java:39) at org.elasticsearch.shield.authc.DefaultAuthenticationFailureHandler.unsuccessfulAuthentication(DefaultAuthenticationFailureHandler.java:35) at org.elasticsearch.shield.authc.InternalAuthenticationService.authenticate(InternalAuthenticationService.java:119) at org.elasticsearch.shield.rest.ShieldRestFilter.process(ShieldRestFilter.java:73) at org.elasticsearch.rest.RestController$ControllerFilterChain.continueProcessing(RestController.java:263) at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:176) at org.elasticsearch.http.HttpServer.internalDispatchRequest(HttpServer.java:128) at org.elasticsearch.http.HttpServer$Dispatcher.dispatchRequest(HttpServer.java:86)

On using 2nd config , getting below:
[2016-05-27 11:14:44,488][DEBUG][shield.authc.ldap user not found in cache, proceeding with normal authentication [2016-05-27 11:14:44,503][DEBUG][shield.authc.ldap ] authentication failed for user [testuser] java.lang.NullPointerException at org.elasticsearch.shield.authc.ldap.UserAttributeGroupsResolver.resolve(UserAttributeGroupsResolver.java:56) at org.elasticsearch.shield.authc.ldap.support.LdapSession.groups(LdapSession.java:77) at org.elasticsearch.shield.authc.ldap.support.AbstractLdapRealm.createUser(AbstractLdapRealm.java:89) at org.elasticsearch.shield.authc.ldap.support.AbstractLdapRealm.doAuthenticate(AbstractLdapRealm.java:51) at org.elasticsearch.shield.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:92) at org.elasticsearch.shield.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:32) at org.elasticsearch.shield.authc.InternalAuthenticationService.authenticate(InternalAuthenticationService.java:363) at org.elasticsearch.shield.authc.InternalAuthenticationService.authenticate(InternalAuthenticationService.java:109) at org.elasticsearch.shield.rest.ShieldRestFilter.process(ShieldRestFilter.java:73)

Below are log lines related to mapped groups:

[2016-05-27 11:01:25,280][DEBUG][shield.authc.support ] the roles [[]], are mapped from these [ldap] groups [[]] for realm [ldap/ldap1]

17

Can you provide a ldif of a user and a group? The values can be obfuscated; I just need to see the available attributes.

I'll open up internal bugs since we should handle the case better when the search doesn't retrun the expected values.

18

checking on it- will provide once i have it,. Thanks.

19

Hey @jaymode
Do you need default query or create or may be you can provide an ldapsearch query - that i can get?

20

Default query should be fine. I think something like this might work:

ldapsearch -b "uid=testuser,ou=People,dc=company,dc=com" -s base
ldapsearch -b "cn=SuperUsers,ou=group,dc=company,dc=com" -s base

Basically you just want to search at that level in the directory and only the base scope.