HTTPS Client Certificate Only Authentication [ES 6.8] [NEST] [SSL/TLS]

[mqb (Matt B)]

Is it possible to authenticate with only a self-signed certificate (without basic auth) via NEST 6.x client to an Elasticsearch 6.8 cluster that has SSL/TLS enabled? If so, what are the requirements (ie. subscription level, realm) necessary and is there a NEST code example to establish the connection?

NEST client request attempts over HTTPS have resulted in a response back from the ES node indicating a basic authentication credential (user/pass) is required.

Sources:

" ClientCertificate
Use a X509Certificate to authenticate all HTTP requests. You can also set them on individual request using ClientCertificates"
-- Elasticsearch NEST Client Configuration Options Source

"[...] one alternative is to use Public Key Infrastructure (PKI) (client certificates) for authenticating to an Elasticsearch cluster."
-- Elastic Blog Source

" The certificate in this case isn't used for certificate authentication to Elasticsearch, as the ClientCertificate method is used for, but is used for Transport Layer Security (TLS)."
-- StackOverflow Source

The cluster was created using Elastic Azure 6.8 ARM template using Certificates and CA created using Elastic utility tool.

[TimV (Tim Vernum)]

@mqb

what are the requirements (ie. subscription level, realm) necessary

Elasticsearch supports authentication via TLS Client Certificates using the PKI realm, which requires a minimum of a Gold (or Trial) license.

@mqb

only a self-signed certificate

I would discourage the use of a self-signed certificate for this. It would be easier to generate your own CA certificate and use that rather than try and configure PKI with a self-signed client cert.

[mqb (Matt B)]

Tim thank you for the additional insights. Seems that there is an info gap or lack of fundamental knowledge on my end regarding the configuration necessary to authentication with just the certificate. The setup is currently designed to use just the enabled ES 6.8 security features (no PKI realm).

To confirm, is it possible to authenticate a client (NEST 6.x) by certificate only - not in addition to - removing the need for basic auth credentials?

@TimV

It would be easier to generate your own CA certificate and use that

We did create a CA cert using the elasticsearch-certutil tool which had then been used in the Azure ARM template for environment creation.

/etc/elasticsearch/elasticsearch.yml

cluster.name: "elasticsearch-tls"
node.name: "data-2"
path.logs: /var/log/elasticsearch
path.data: /datadisks/disk1/elasticsearch/data
discovery.zen.ping.unicast.hosts: ["master-0:9300","master-1:9300","master-2:9300"]
discovery.zen.minimum_master_nodes: 2
node.master: false
node.data: true
network.host: [_site_, _local_]
node.max_local_storage_nodes: 1
node.attr.fault_domain: 1
node.attr.update_domain: 1
cluster.routing.allocation.awareness.attributes: fault_domain,update_domain
azure.client.default.endpoint_suffix: core.windows.net
xpack.license.self_generated.type: trial
xpack.security.enabled: true
bootstrap.memory_lock: true
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: /etc/elasticsearch/ssl/elasticsearch-http.p12
xpack.security.http.ssl.truststore.path: /etc/elasticsearch/ssl/elasticsearch-http.p12
xpack.security.http.ssl.verification_mode: certificate
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/ssl/elasticsearch-transport.p12
xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/ssl/elasticsearch-transport.p12

[TimV (Tim Vernum)]

@mqb

To confirm, is it possible to authenticate a client (NEST 6.x) by certificate only - not in addition to - removing the need for basic auth credentials?

It is certainly possible to authenticate "a client" by certificate only. I cannot see a reason why that would not be possible in NEST, but I'm not a .NET engineer, so I've never done it myself.

The high level steps you need are:

1. Turn on client authentication for HTTP. e.g.

   xpack.security.http.ssl.client_authentication: optional
   

2. Enable a PKI Realm:

   xpack.security.authc.realms:
     file1:
       type: file
       order: 1
     native1:
       type: native
       order: 2
     pki1:
       type: pki
       order: 3
   

3. Connect using a client certificate that is trusted by your HTTP ssl config (in this case, a certificate that was signed by the same CA as you used to generate elasticsearch-http.p12)

[forloop (Russ Cam)]

With NEST or Elasticsearch.Net, you can use a certificate for authentication by using the .ClientCertificate(...) method on ConnectionSettings/ConnectionConfiguration, respectively.

For example,

var settings = new ConnectionSettings(pool)
    .ClientCertificate(@"C:\path_to_cert");

var client = new ElasticClient(settings);

or if you need more control over how an X509Certificate instance is created

var settings = new ConnectionSettings(pool)
    .ClientCertificate(new X509Certificate2(@"C:\path_to_cert"));

var client = new ElasticClient(settings);

Take a look at the .NET client documentation on Client Certificates.