Elastic LDAP config issues

Hello, I’m working on setting up Ldap/AD auth. However, I’m not sure what I’m missing here. In terms of logging, I’m not seeing much of anything in Kibana or Elasticsearch at all. Mainly due to the logs getting clobbered by “not starting watcher, upgrade API run required: .watches [false], .triggered_watches[true]” secondary issue (working on it). I do know that my node can see the LDAP server. I have the proper binduser and password in place.

We have two user groups created in AD (admins and user) with admin tied to my member of and this seems to be setup correctly in my role_mapping.yml.

However, when I try to log in, all I’m getting is Invalid username or password. Please try again.

   "version" : {
    "number" : "7.9.1",

elasticsearch.yml

# ======================== Elasticsearch Configuration =========================
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
# ---------------------------------- Cluster -----------------------------------
# Use a descriptive name for your cluster:
cluster.name: elastic_int
# ------------------------------------ Node ------------------------------------
# Use a descriptive name for the node:
node.name: xyz
# Add custom attributes to the node:
node.attr.datacenter: xyz
# ----------------------------------- Paths ------------------------------------
# Path to directory where to store the data (separate multiple locations by comma):
path.data: /mnt/data
# Path to log files:
path.logs: /var/log/elasticsearch
# ----------------------------------- Memory -----------------------------------
# Lock the memory on startup:
# bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
# Elasticsearch performs poorly when the system is swapping the memory.
# ---------------------------------- Network -----------------------------------
# Set the bind address to a specific IP (IPv4 or IPv6):
network.host: 0.0.0.0
#http.port: 9200
#
# --------------------------------- Discovery ----------------------------------
discovery.seed_hosts: ["0.0.0.0","0.0.0.0","0.0.0.0"]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#cluster.initial_master_nodes: ["node1"]
# ---------------------------------- Various -----------------------------------
#

# At this % disk usage, elasticsearch will mark all indices read-only (really delete-only)
cluster.routing.allocation.disk.watermark.low: 90%
cluster.routing.allocation.disk.watermark.high: 95%
cluster.routing.allocation.disk.watermark.flood_stage: 98%


xpack.monitoring.collection.enabled: true
xpack.monitoring.history.duration: 30d

xpack.security.enabled: true
#xpack.security.audit.enabled: true
#cluster ssl
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
#client ssl
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: x.x.p12

thread_pool.write.queue_size: 1000
indices.memory.index_buffer_size: 15%

#LDAP config
    xpack:
      security:
        authc:
          realms:
            ldap:
              ldap1:
                order: 0
                url: "ldap://x.x.x.x"
                bind_dn: "cn=binduser, ou=ds-service-accounts, dc=int, dc=tt, dc=local"
                user_search:
                  base_dn: "dc=idx,dc=id,dc=local"
                  filter: "(cn={0})"
                group_search:
                  base_dn: "dc=idx,dc=id,dc=local"
                files:
                  role_mapping: "/etc/elasticsearch/role_mapping.yml"
                unmapped_groups_as_roles: false 

role_mapping.yml

    role # Role mapping configuration file which has elasticsearch roles as keys
    # that map to one or more user or group distinguished names

    #roleA:   this is an elasticsearch role
    #  - groupA-DN  this is a group distinguished name
    #  - groupB-DN
    #  - user1-DN   this is the full user distinguished name

    superuser:
       - "cn=Kibana-Admins,ou=Applications-Group,ou=Groups,dc=idx,dc=id,dc=local"
    user:
       - "cn=Kibana-Users,ou=Applications-Group,ou=Groups,dc=idx,dc=id,dc=local"
    #  - "cn=admins,dc=example,dc=com"
    #  - "cn=John Doe,cn=other users,dc=example,dc=com"
    elasticsearch]$ curl --user user:pw.. -X GET "http://x.x.x.x:9200?pretty" --verbose
    * About to connect() to proxy xyz port 3128 (#0)
    *   Trying x.x.x.x...
    * Connected xyz.net (0.0.0.0) port xyz (#0)
    * Server auth using Basic with user 'bind_dn'
    > GET http://0.0.0.0:9200/?pretty HTTP/1.1
    > Authorization: Basic YmluZF9kbjpUdDEyMzQ1Njc4Li4=
    > User-Agent: curl/7.29.0
    > Host: x.x.x.x:9200
    > Accept: */*
    > Proxy-Connection: Keep-Alive

Hey @dmartinezz23,

The curl output you provided at the end looks incomplete. Were you able to get a response back from ES?

Can you try hitting the http://x.x.x.x:9200?_security/_authenticate endpoint instead with those same credentials? This will return information about your user, so we can make sure that your roles are getting mapped the way you expect.

Once we've done all that, can you enable debug logging for Kibana, try to log in to Kibana and provide those logs to us? These debug logs will show the authentication attempts to ES in more detail, and that can help us debug further.

# kibana.yml
logging.verbose: true
logging.quiet: false

Just now cathcing this in my logging

[2020-10-22T13:13:25,546][DEBUG][o.e.x.s.a.AuthenticationService] [xyz] Authentication of [dmartinez] using realm [ldap/ldap1] withtoken [UsernamePasswordToken] was [AuthenticationResult{status=CONTINUE, user=null, message=null, exception=null}]
org.elasticsearch.ElasticsearchSecurityException: unable to authenticate user [xyz] for REST request [/_security/_authenticate]
org.elasticsearch.ElasticsearchSecurityException: unable to authenticate user [xyz] for REST request [/_security/_authenticate]
[2020-10-23T10:35:33,627][DEBUG][o.e.x.s.a.AuthenticationService] [] Authentication of [dmartinez] using realm [ldap/ldap1] with token [UsernamePasswordToken] was [AuthenticationResult{status=CONTINUE, user=null, message=authenticate failed, exception=LDAPException(resultCode=49 (invalid credentials), errorMessage='invalid credentials', ldapSDKVersion=4.0.8, revision=28812)}]
org.elasticsearch.ElasticsearchSecurityException: unable to authenticate user xyz] for REST request [/_security/_authenticate]
org.elasticsearch.ElasticsearchSecurityException: unable to authenticate user [xyz] for REST request [/_security/_authenticate]


I Think this will be easier to troubleshoot on the elasticsearch side than on the kibana side, as elasticsearch handles all the LDAP functionality.

  1. Have you added xpack.security.authc.realms.ldap.ldap1.secure_bind_password in the elasticsearch keystore ?

  2. Is the password correct? i.e. can you bind with cn=binduser, ou=ds-service-accounts, dc=int, dc=tt, dc=local and that password to your ldap server using i.e. ldapsearch or some other tool ?

  3. filter: "(cn={0})" means that whatever you are entering as a username in curl or kibana login form should be the CN of that user you want to login with, is dmartinez the CN of your user in ldap ?

Please share a large portion of the logs, at least one that contains the whole authentication process and not a single line. Also please share the request that yields that response and if it is necessary to sanitize/mask data, do so consistently so that we can follow along!

Alright,

verified that LDAP authentication is working correctly to Elastic (see log entry below)
We however have not been able to get kibana to play nice about logging LDAP users into a workspace. We’re getting a consistent error that we’re still chasing (see error message below)

Item 1
----
[2020-10-23T17:35:03,123][DEBUG][o.e.x.s.a.AuthenticationService] [xyz] Authentication of [dm] using realm [native/navtive1] with token [UsernamePasswordToken] was [AuthenticationResult{status=SUCCESS, user=User[username=joshtt,roles=[tt_admin],fullName=Dm Test Account,email=Dm.email,metadata={}], message=null, exception=null}]
Item 3
----
[2020-10-23T17:35:22,364][DEBUG][o.e.x.s.a.AuthenticationService] [xyz] Authentication of [dm] using realm [ldap/ldap1] with token [UsernamePasswordToken] was [AuthenticationResult{status=SUCCESS, user=User[username=dm,roles=[superuser]
Item 4
----
{"statusCode":401,"error":"Unauthorized","message":"[security_exception] unable to authenticate user [dm] for REST request [/_security/_authenticate], with { header={ WWW-Authenticate={ 0=\"Bearer realm=\\\"security\\\"\" & 1=\"ApiKey\" & 2=\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\" } } }"}

ouput from my curl command

{
  "name" : "xyz",
  "cluster_name" : "elastic_int",
  "cluster_uuid" : 
  "version" : {
    "number" : "7.9.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : 
    "build_date" : "2020-09-01T21:22:21.964974Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

The curl command confirmed that we can query the API with our LDAP credentials as well. I should also mention that we are running one node with Kibana/elasticsearch and two nodes with just elasticsearch with matching configs.

I see one specific log entry for each case again, without any other relevant entries. These 2 lines show the same user authenticating once in the native realm ( which given the configuration you have shared, shouldn't even be enabled now ) and once in the ldap realm, so I'm not really sure what to read into this. You still need to add more information about your setup or consistently obfuscate your output, as asked above.

You share an error line but you don't tell us how you got that error line, where and doing what. What's more , it's the same user that we see successfully authenticating above. I guess this is a third user and you have changed it's username to 'dm' here too ? It's going to be really hard to help out if we keep on guessing.

You say that the error is reproducible so can you please:

  • Enable DEBUG logging in elasticsearch. It looks like you have this already but just making sure.
  • Enable DEBUG iogging in kibana as Larry asked you above.
  • Perform the action that gets you the error ( I guess logging in via kibana ? )
  • Provide the logs ( all the logs not just the line you think is helpful ) from that couple of seconds from elasticsearch and kibana.
  • Provide us the current configuration related to realms from your elasticsearch.yml
  • If you need to obfuscate data, please do so in a consistent way.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.