Can't configure PKI authentication

Elastic Stack Elasticsearch
1

Hey all, I'm trying to use PKI authentication in my cluster, meaning the client will need to present certificates in order to get data.
I wrote the following in elasticsearch.yml:
xpack: security: authc: realms: pki1: type: pki

And removed my "native" realm.

I changed nothing else in my code (c#). Right now, I can still get data using my "elastic" user, and if I don't use it I get"missing authentication token for REST request".

Unless I don't understand something, this seems wrong, as I'm expecting a security exception of some sort.
Thank you!

EDIT:
I tried setting xpack.security.http.ssl.client_authentication: required.
This changes everything. Now I'm getting:
'The underlying connection was closed: An unexpected error occurred on a send.'

Which is good, I think. I generated a certificate for my client using certgen, and used it to generate an x509Certificate in C#, I also added the autogenerated CA that signed it to xpack.ssl.certificate_authorities, and mapped the CN to a role. Still getting this error. Installing the client's CA on the elastic machine didn't help either.

2

Hi @Shlomi_Toren

Where possible please use the code button (</>) when pasting in your config files so that it retains the correct formatting. Thanks.

There's a few things going on here:

  • The elastic user isn't part of the native realm, so removing that realm won't change whether the elastic user works. (More Info)
  • Simply enabling a PKI realm doesn't force that users need to use TLS/PKI. If you want to enforce client certificates you need to do that at the http layer with the xpack.security.http.ssl.client_authentication setting . (More Info)
  • You say I'm expecting a security exception but the error message you received (missing authentication token for REST request) is a security exception (on the Elasticsearch side, I haven't looked at the C# side) . Can you clarify how that message doesn't meet your expectation?

A working configuration for PKI would typically look something like this:

xpack.security.authc.realms:
  pki1:
    type: pki
    order: 0
    certificate_authorities: tls/client/ca.crt
    files.role_mapping: pki-roles.yml

xpack.ssl:
  certificate: tls/server/node01/node01.crt
  key: tls/server/node01/node01.key

xpack.security.http.ssl:
  enabled: true
  client_authentication: optional
  certificate_authorities: [ "tls/client/ca.crt" ]

xpack.security.transport.ssl:
  enabled: true
  certificate_authorities: [ "tls/server/ca/ca.crt" ]

That said, there's lots of different configuration options, so your config will almost certainly look different, but that's the rough approach.

3

I'll try this and see if it works, thanks.

Just to clarify: The fact that the elastic user is NOT a part of the native realm, means I HAVE to use it with PKI?

4

xpack.ssl.certificate is the certificate that the server provides to incoming connections (either HTTP or transport).

You can't simply use a CA for that purpose as the server needs to have its own certificate configured in some way.

You can use a keystore instead of a certificate, but you need one of them.

5

You don't have to use it. But removing the native realm doesn't change whether that user is available or not.

As a general rule your code shouldn't ever use the elastic user. It's a superuser, roughly equivalent to root on a unix system. Your code should be using it's own user (either representing the person logged in, or the application itself, depending on your security mode). Whether you authenticate that user via password or TLS certificate is up to you.

6

Of course, I confused it for something else. Sorry.

7

Ok, to be more specific: Do I need to specify a user in the request even if I'm using a certificate? And should that user be the CN of the certificate?

8

No, if you authenticate via certificate you do not need to provide a user.

For example this curl command (my apologies for not having a more windows-oriented example) will successfully authenticate a user using only a certificate (the password supplied is the passphrase on the certificate file, it's not sent to the server).

curl --cert my-cert.p12:cert-password \
     --cacert server/ca.crt \
     'https://localhost:9200/_xpack/security/_authenticate?pretty'
9

Still didn't cut it. I guess I need to show you some code now, but the problem is I have an infrastructure for other types of REST applications that I'm using, and I'm using RESTSharp in it. Here's my code:

System.Net.NetworkCredential c = new System.Net.NetworkCredential("elastic", "password!");
httpRequest.Credentials = c;

X509Certificate cert = new X509Certificate(@"path\client1.crt");

httpRequest.ClientCertificates = new X509CertificateCollection();
httpRequest.ClientCertificates.Add(cert);

The paths I'm using are correct, as I can generate the certificate, and the rest of this code works fine without PKI. Do you have an idea as to what could go wrong?

10

What format is client1.crt? Typically a .crt file is only the public side of the certificate, and you need a separate key file that contains the private key.

It's possible to have a combined PKCS-12 formatted key, but I'd double check your file formats.

11

I see. certgen generated this file along with a .key file. How do I use both of them in this context?
Same goes for your curl command. I'm trying it and getting
curl: (58) unable to set private key file: 'client1.crt' type PEM
So I guess I do have a problem with the .crt file.

12

I'm not a .NET dev, but I believe X509Certificate requires a combined certificate in PKCS-12 or pfx format.

If you have openssl available you can combine the .crt and .key files into a .p12 with:

openssl pkcs12 -export -clcerts -in client1.crt -inkey client1.key -out client1.p12

I think pvk2pfx does something similar on windows, but I can't offer any more advice than that.

13

Cool, I was looking for the openssl command but couldn't find it, thanks. I packaged the certificate and key, gave a password, and gave the X509Certificate object the p12 file and the password. Still didn't work.

Can you think about anything else that might cause my issue? Maybe the certificate itself? I gave it the IP and DNS name of the client that's running it, but I don't know if ES even checks that.

14

Holy hell. Turns out, there's also a X509Certificate2 object. It just worked. I guess that solves it.
Thank you so much for your help TIm!

15

Glad you got it sorted out @Shlomi_Toren :thumbsup:

We also have documentation for working with certificates with NEST, with a helper class, ClientCertificate to make it a little easier to set up on the client (.NET Framework only, as .NET Core does not allow the PrivateKey property to be set on the crypto service provider that is used to generate the pfx file at runtime.)

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.