Setting SSL Between Elasticsearch and Active Directory [ELK running on Windows server]

Elastic Stack Elasticsearch
1

Hi Team,

I've been working on sort of unorthodox ELK stack logging POC solution where all components (ElasticSearch, Kibana, Logstash, X-Pack) are running on windows server.

I've been able to integrate Elastic search with a custom active directory realm and users are able to login to Kibana with their NT accounts however when I try to secure this communication via ldaps on port 636, I am hitting a problem

I have installed a digital certificate on the AD domain controller and imported the certificate in the ELK node server into the Trusted Root Certification Authorities. Tested that ldaps connection from the ELK server to the domain controller is enabled via the ldp.exe tool:

ld = ldap_sslinit("dc01.xxxxx.xxxx", 636, 1);
Error 0 = ldap_set_option(hLdap, LDAP_OPT_PROTOCOL_VERSION, 3);
Error 0 = ldap_connect(hLdap, NULL);
Error 0 = ldap_get_option(hLdap,LDAP_OPT_SSL,(void*)&lv);
Host supports SSL, SSL cipher strength = 256 bits
Established connection to dc01.devops.test.
Retrieving base DSA information...

As per the official article my elasticsearch.yml is configured as follows:

#----------------------Configuration for X-Pack Security------------------------

xpack:
    security:
        authc:
            realms:
                active_directory:
                    type: "active_directory"
                    order: 0
                    domain_name: "xxxxx.xxxx"
                    url: "ldaps://dc01.xxxxxx.xxxx:636"
                    unmapped_groups_as_roles: true

What could I be possibly missing?

The elasticsearchlog shows the following errors when I try to access kibana

[WARN ][o.e.x.s.a.AuthenticationService] [B-ChoWB] Authentication to realm active_directory failed - authenticate failed (Caused by LDAPException(resultCode=91 (connect error), errorMessage='An error occurred while attempting to connect to server dc01.devops.test:636: java.io.IOException: LDAPException(resultCode=91 (connect error), errorMessage='Unable to verify an attempt to to establish a secure connection to 'dc01.devops.test:636' because an unexpected error was encountered during validation processing: SSLPeerUnverifiedException(message='peer not authenticated',

I've tried to add the following to the yml configuration but then the elasticsearch service will not start and probably failing to parse the configuration

ssl:
certificate_authorities: [ "CONFIG_DIR\x-pack\cacert.pem" ]

I couldn't find examples on how this configuration should look like on a windows server. How does X-Pack know where is the default configuration directory or should I set this up anywhere?

Thanks a lot!

p.s. excuse me for the long post

2

On the contrary, detailed posts are far easier to understand and solve than the ones that leave out all the important details. :slight_smile:

What exactly do you mean by that? The JVM has its own trusted certificates and does not read the windows trusted certs, so depending on exactly whrere you imported it, it may have had no effect on ES.
Based on the fact that you tested it with ldp.exe, I assume you added it to the OS store, not the JVM store.

The syntax you want (untested) is:

 xpack:
    security:
        authc:
            realms:
                active_directory:
                    type: "active_directory"
                    # ... other config
                    ssl:
                     certificate_authorities: [ "x-pack/cacert.pem" ]
3

Thanks, TimV

Re: certificates - Yes, the certificate was imported into the windows OS store only.

As per Microsoft documentation for enabling LDAPS, Certificate must be valid for the purpose of Server Authentication. This means that it must also contains the Server Authentication object identifier (OID): 1.3.6.1.5.5.7.3.1

I was planning to use certgen to generate such certificate but I am not sure if it will be suitable for this purpose (having the correct OID, etc..).

Is it possible to use certgen.bat to only import this generated certificate so that elasticsearch will account for it?

Re: yml syntax: - on windows servers do I specify paths in the yml configuration with forward or back slashes? Initially I did backslashes to follow the windows style :smiley:

Last question, how does ElasticSearch or X-Pack know where is the default config directory set. For example this syntax [ "x-pack/cacert.pem" ] : how dows it know where to find "x-pack" directory? Probably I need to set this up in a system variable..?

For the installation I used the classic offline method with the zip files and not the MSI, so my configuration directory is currently "E:\elasticsearch\config\x-pack". In this case, should I give the absolute path to any items that I would like to use there (like roles.yml, etc..)

i.e.
ssl:
certificate_authorities: [ "E:/elasticsearch/config/x-pack/cert.cer" ]

Thanks in advance!

--Boz

4

Just a quick update. I tried with TimV's eaxmple for the configuration and elastic was able to read the configuration correctly as is:

xpack:
    security:
        authc:
            realms:
                active_directory:
                    type: "active_directory"
                    order: 0
                    domain_name: "xxxxx.xxxxx"
                    url: "ldaps://dc01.xxxxx.xxxxx:636"
                    unmapped_groups_as_roles: true
                    ssl:
                        certificate_authorities: [ "x-pack/ldps.cer" ]

It looks like placing the pem encoded certificate in the x-pack configuration directory is sufficient

I was able to access kibana with an LDAP user with the above settings in the elasticsearch.yml file.

Many thanks, Tim :slight_smile:

5

Either will work, but backslashes can have additional meanings in configurations, so I recommend using forward slashes everywhere.

Unfortunately X-Pack is not 100% consistent, but most of the time files are relative to the directory that contains your elasticsearch.yml file. (More accurately, ES has the concept of a config directory, and it looks for your elasticsearch.ymlin that directory, and X-Pack treats SSL paths as being relative to that directory as well.)

It's personal preference really. On the one hand, with absolute paths you know exactly which file is being read. On the other hand, it means your config is tied to your current directory structure.
I always use relative paths because I have 20+ different ES versions installed on my computer, and I like to be able to copy pieces of config between them.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.