Unable to connect elasticsearch with ldap realm

Hi.

We are planing to migrate our elasticseach security from Searchguard to xpac security.

Below is the error we are getting while connecting to elasticseach via ldap realm

[2020-05-06T04:46:53,557][DEBUG][o.e.x.s.a.AuthenticationService] [elk-dev9] Authentication of [xprl187] using realm [reserved/reserved] with token [UsernamePasswordToken] was [AuthenticationResult{status=CONTINUE, user=null, message=null, exception=null}]
[2020-05-06T04:46:57,121][DEBUG][o.e.x.s.a.l.LdapRealm    ] [elk-dev9] Exception occurred during authenticate for ldap/ldap1
com.unboundid.ldap.sdk.LDAPBindException: NDS error: failed authentication (-669)
    at com.unboundid.ldap.sdk.LDAPConnectionPool.createConnection(LDAPConnectionPool.java:1372) ~[unboundid-ldapsdk-4.0.8.jar:4.0.8]
    at com.unboundid.ldap.sdk.LDAPConnectionPool.createConnection(LDAPConnectionPool.java:1258) ~[unboundid-ldapsdk-4.0.8.jar:4.0.8]
    at com.unboundid.ldap.sdk.LDAPConnectionPool.getConnection(LDAPConnectionPool.java:1792) ~[unboundid-ldapsdk-4.0.8.jar:4.0.8]
    at java.security.AccessController.doPrivileged(AccessController.java:554) ~[?:?]
    at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.privilegedConnect(LdapUtils.java:74) ~[x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.searchForEntry(LdapUtils.java:261) [x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.xpack.security.authc.ldap.support.LdapUtils.searchForEntry(LdapUtils.java:213) [x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.xpack.security.authc.ldap.LdapUserSearchSessionFactory.findUser(LdapUserSearchSessionFactory.java:222) [x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.xpack.security.authc.ldap.LdapUserSearchSessionFactory.getSessionWithPool(LdapUserSearchSessionFactory.java:79) [x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.xpack.security.authc.ldap.PoolingSessionFactory.session(PoolingSessionFactory.java:96) [x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.xpack.security.authc.ldap.LdapRealm.lambda$doAuthenticate$1(LdapRealm.java:131) [x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.xpack.security.authc.ldap.LdapRealm$CancellableLdapRunnable.doRun(LdapRealm.java:314) [x-pack-security-7.6.1.jar:7.6.1]
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692) [elasticsearch-7.6.1.jar:7.6.1]
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.6.1.jar:7.6.1]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
    at java.lang.Thread.run(Thread.java:830) [?:?]
[2020-05-06T04:46:57,130][DEBUG][o.e.x.s.a.AuthenticationService] [elk-dev9] Authentication of [XXXXX] using realm [ldap/ldap1] with token [UsernamePasswordToken] was [AuthenticationResult{status=CONTINUE, user=null, message=authenticate failed, exception=LDAPException(resultCode=49 (invalid credentials), diagnosticMessage='NDS error: failed authentication (-669)', ldapSDKVersion=4.0.8, revision=28812)}]
[2020-05-06T04:46:57,130][WARN ][o.e.x.s.a.AuthenticationService] [elk-dev9] Authentication to realm ldap1 failed - authenticate failed (Caused by LDAPException(resultCode=49 (invalid credentials), diagnosticMessage='NDS error: failed authentication (-669)', ldapSDKVersion=4.0.8, revision=28812)) 

NOTE: Same ldap configuartion is working fine with SeachGuard Ldap setup.

LDAP Realm Configuration

xpack.security:
      authc:
          realms:
              ldap:
                  ldap1:
                    order: 3
                    url: [ "ldap://xxxxx.ldp.tla.xxx.com:389" ]
                    ssl.certificate_authorities: [ "ca-bundle.cer" ]
                    bind_dn: 'cn=xxxx,ou=xxr,ou=xx'
                    bind_password: '****'
                    user_search:
                      base_dn: 'ou=xxx,ou=xx'
                      filter: '(member={0})'
                    group_search:
                      base_dn: "ou=xx,ou=xx"
                      user_attribute: cn
                    files:
                      role_mapping: "<path>/roles_mapping.yml"
                    unmapped_groups_as_roles: true
                    ssl.key.path: xxx.key
                    ssl.certificate: ca-bundle.cer
                    authentication.enabled: true

The error message that we get shows that a password is incorrect : https://support.novell.com/docs/Tids/Solutions/10067240.html. Can you verify that

a) Your bind_dn and bind_password are actually correct, via an ldapsearch for example.
b) Enable trace logging for LDAP

 curl -H "Content-Type: application/json" -XPUT -uelastic 'http://<instance>:9200/_cluster/settings' -d'
{
   "transient" : {
     "logger.org.elasticsearch.xpack.security.authc.ldap" : "TRACE"
 }
}' 

In the meantime,

user_search:
    base_dn: 'ou=xxx,ou=xx'
    filter: '(member={0})'

seems very strange to me. That filter is used to find users under ou=xxx,ou=xx and it would seem improbably that your user entries have a member attribute that references their username. See here on what filter means exactly.

Also

group_search:
    base_dn: "ou=xx,ou=xx"
    user_attribute: cn

seems rather strange too as it implies that you have groups with the group , groupOfNames or groupOfUniqueNames objectclass that contain a list of members by cn. This is uncommon, but then again, you know your setup better than us.

NOTE: Same ldap configuartion is working fine with SeachGuard Ldap setup.

I am not aware of how the ldap realm is configured in this product but I can only assume there is no 1-1 exact mapping for configuration that you can port over to elasticsearch as-is.

@ikakavas need your help.

I am configuring LDAP realm, but LDAP groups are not getting resolved. Please find below my LDAP realm configuration.

xpack.security:
      authc:
          realms:
              ldap:
                  ldap1:
                    order: 3
                    url: [ "ldap://devxx-hq.ldp.tla.xxx.com:389","ldap://prdxx-dr.ldp.tla.xxx.com:8389" ]
                    ssl.certificate_authorities: [ "ca-bundle.cer" ]
                    bind_dn: "cn=xxxxx,ou=xx,o=xx"
                    bind_password: "xxxxx"
                    user_search:
                      base_dn: "ou=xxxx,o=xx"
                      filter: "(cn={0})"
                    group_search:
                      base_dn: "ou=xxxx,o=xx"
                      filter: "(uid={0})"
                      user_attribute: uid
                    unmapped_groups_as_roles: true

elasticsearch logs

[2020-06-02T06:16:10,953][TRACE][o.e.x.s.a.l.s.LdapUtils  ] [CLIENT-NODE] LDAP Search SearchRequest(baseDN='ou=xxxx,o=xx', scope=SUB, deref=NEVER, sizeLimit=0, timeLimit=5, filter='(cn=xxxx187)', attrs={uid}) => SearchResult(resultCode=0 (success), messageID=2, entriesReturned=1, referencesReturned=0) ([SearchResultEntry(dn='cn=xxxx187,ou=xxxx,o=xx', messageID=2, attributes={Attribute(name=uid, values={'xxxx187'})}, controls={})])
[2020-06-02T06:16:10,975][TRACE][o.e.x.s.a.l.s.LdapUtils  ] [CLIENT-NODE] LDAP bind [SimpleBindRequest(dn='cn=xxxx187,ou=xxxx,o=xx')] succeeded for [LDAPConnectionPool(serverSet=FailoverServerSet(serverSets={SingleServerSet(server=xxxapp01-hq.ldp.tla.xxxx.com:389, includesAuthentication=false, includesPostConnectProcessing=false), SingleServerSet(server=xxxx-dr.ldp.tla.xxxx.com:8389, includesAuthentication=false, includesPostConnectProcessing=false)}), maxConnections=20)]
[2020-06-02T06:16:10,976][DEBUG][o.e.x.s.a.l.LdapUserSearchSessionFactory] [CLIENT-NODE] Resolving LDAP groxxs + meta-data for user [cn=xxxx187,ou=xxxx,o=xx]
[2020-06-02T06:16:10,981][TRACE][o.e.x.s.a.l.s.LdapUtils  ] [CLIENT-NODE] LDAP Search SearchRequest(baseDN='ou=xxxx,o=xx', scope=SUB, deref=NEVER, sizeLimit=0, timeLimit=5, filter='(uid=xxxx187)', attrs={1.1}) => SearchResult(resultCode=0 (success), messageID=5, entriesReturned=1, referencesReturned=0) ([SearchResultEntry(dn='cn=xxxx187,ou=xxxx,o=xx', messageID=5, attributes={}, controls={})])
[2020-06-02T06:16:10,981][DEBUG][o.e.x.s.a.l.LdapUserSearchSessionFactory] [CLIENT-NODE] Resolved 1 LDAP groxxs [[cn=xxxx187,ou=xxxx,o=xx]] for user [cn=xxxx187,ou=xxxx,o=xx]
[2020-06-02T06:16:10,982][DEBUG][o.e.x.s.a.l.LdapUserSearchSessionFactory] [CLIENT-NODE] Resolved 0 meta-data fields [{}] for user [cn=xxxx187,ou=xxxx,o=xx]
[2020-06-02T06:16:10,998][TRACE][o.e.x.s.a.s.m.NativeRoleMappingStore] [CLIENT-NODE] Applying role-mapping [ldap-example] to user-model [{groxxs=[cn=xxxx187,ou=xxxx,o=xx], realm.name=ldap1, dn=cn=xxxx187,ou=xxxx,o=xx, metadata.ldap_dn=cn=xxxx187,ou=xxxx,o=xx, username=xxxx187, metadata.ldap_groxxs=[cn=xxxx187,ou=xxxx,o=xx]}] produced role-names [[sxxeruser]]
[2020-06-02T06:16:10,998][DEBUG][o.e.x.s.a.s.m.NativeRoleMappingStore] [CLIENT-NODE] Mapping user [UserData{username:xxxx187; dn:cn=xxxx187,ou=xxxx,o=xx; groxxs:[cn=xxxx187,ou=xxxx,o=xx]; metadata:{ldap_dn=cn=xxxx187,ou=xxxx,o=xx, ldap_groxxs=[cn=xxxx187,ou=xxxx,o=xx]}; realm=ldap1}] to roles [[sxxeruser]]
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 469
<
{
  "cluster_name" : "XXXXXXXXXX1",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 7,
  "number_of_data_nodes" : 3,
  "active_primary_shards" : 21,
  "active_shards" : 42,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
* Connection #0 to host localhost left intact

I have created a role-mapping with LDAP group ORG-ADMIN but same group is not getting resolved by LDAP, however if I created a role mapping as "cn=xxxx,ou=xxx,o=o" its getting resolved.

Regards/-
Harsh

Please don't ping folks directly, this is a public forum and anyone could answer a question. If you are in need of a service with an SLA that covers response times for questions then you may want to consider talking to us about a subscription or talk to your support engineer if you already have one.

This is not enough information for anyone to help you, unfortunately. Please share the role mapping that works, the role mapping that doesn't work and logs from each case, along with a proper explanation on what we're looking at and why this is not what you expect. The more time you put into your question, the easier it is for someone to assist you, thus the quicker you'll get an answer or help.

Understood, i will share complete information related to role mapping and realm settings.
Regards/-
Harah