Configure PKI authentication for Elasticsearch 7.7

This guide Elasticsearch Security: Configure TLS/SSL & PKI Authentication | Elastic Blog shows how to configure PKI for Elasticsearch version 6.5 however this doesn't work for version 7.7.0 ( the version I'm currently using ). When I add xpack.security.authc.realms.pki1.type: pki to my elasticsearch.yml file, I get the error below:

java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.security.Security] Likely root cause: java.lang.IllegalArgumentException: Incorrect realm settings found. Realm settings have been changed to include the type as part of the setting key. For example 'xpack.security.authc.realms.file.my_file.order' Found invalid config: xpack.security.authc.realms.pki1.type Please see the breaking changes documentation.

I went through the documentation for Elasticsearch 7.7 but didn't find anything specific to PKI changes. Can someone please assist with the settings to use and also how to generate the client certificate for authentication? Thanks

Elasticsearch 7.7 is EOL and no longer supported. Please upgrade ASAP.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

Does this help?

Add a realm configuration for a pki realm to elasticsearch.yml under the xpack.security.authc.realms.pki namespace.

Thanks for the reply. Using the settings below works. That is Elasticsearch now starts up without an error. Would you know how I can configure the client and server certificates to use for mutual authentication?

xpack:
  security:
    authc:
      realms:
        pki:
          pki1:
            order: 1

After further research, I was able to use the certutil command to generate a client cert. Elasticsearch however still seems to require a username and password.

Command

bin/elasticsearch-certutil cert --ca-cert certs/ca/ca.crt --ca-key certs/ca/ca.key -out client.p12

I then used this client.p12 cert to authenticate to the cluster

curl -k --cert-type P12 --cert client.p.12 https://localhost:9200

I get a 401 error

{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}%

I'm able to authenticate when I provide the username and password

 curl -k --cert-type P12 --cert client.p.12 https://localhost:9200  -u user:password
{
  "name" : "node01",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "W0MJXDsnTw2LlnhGHw3Lug",
  "version" : {
    "number" : "7.7.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "81a1e9eda8e6183f5237786246f6dced26a10eaf",
    "build_date" : "2020-05-12T02:01:37.602180Z",
    "build_snapshot" : false,
    "lucene_version" : "8.5.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

How do I connect to Elasticsearch with the certificate only and NOT provide a username and password? My elasticsearch.yml is below

node.name: node01
xpack.security.enabled: true
discovery.type: single-node
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/node01.key
xpack.security.transport.ssl.certificate: certs/node01.crt
xpack.security.transport.ssl.certificate_authorities: [ "certs/ca.crt" ]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/node01.key
xpack.security.http.ssl.certificate: certs/node01.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.security.http.ssl.client_authentication: optional

Thanks for your help.

I've still not found a solution to Elasticsearch still requiring basic auth despite trying to use a certificate to authenticate. What setting(s) do I have to add to elasticsearch.yml to make cert authentication work?

What is the complete content of your elasticsearch.yml file? The one you shared immediately above does not include the PKI realm settings.

PKI realm is a license feature. So you'll need either a platinum license or trial license. Otherwise, the PKI realm will be skipped.

This is the entire elasticsearch.yml file

node.name: node01
xpack.security.enabled: true
discovery.type: single-node
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/node01.key
xpack.security.transport.ssl.certificate: certs/node01.crt
xpack.security.transport.ssl.certificate_authorities: [ "certs/ca.crt" ]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/node01.key
xpack.security.http.ssl.certificate: certs/node01.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.security.http.ssl.client_authentication: required
xpack.security.http.ssl.verification_mode: certificate



xpack:
  security:
    authc:
      realms:
        pki:
          pki1:
            order: 1
            certificate_authorities: [ "certs/ca.crt" ]

when I try to connect without the -k flag I get the error below

curl --cert-type P12 --cert client_node.p12 https://localhost:9200

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

and in verbose mode

curl --cert-type P12 --cert client_node.p12 https://localhost:9200 -vvv
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

which certificate is curl trying to verify and is the command I'm using the right way to connect using SSL authentication?

The error means the client (curl) does not trust the server's certificate. It has nothing to do with PKI realm. In the verbose log it says

CAfile: /etc/ssl/cert.pem

I don't think your ES server cert is in that location. So you might want to consider specify --cacert option for curl.

Thanks for your help so far. So I added the --cacert option and I was able to get past the unable to get local issuer certificate error but curl still wants to use basic auth. Is there a setting or a way to ensure only certificate authentication works and basic auth is disabled?

curl --cert-type P12 --cert client_node.p12 https://localhost:9200 --cacert config/certs/ca.crt -vvv
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: config/certs/ca.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=node01
*  start date: Nov 10 01:17:25 2022 GMT
*  expire date: Nov  9 01:17:25 2025 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
> GET / HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Bearer realm="security"
< WWW-Authenticate: ApiKey
< WWW-Authenticate: Basic realm="security" charset="UTF-8"
< content-type: application/json; charset=UTF-8
< content-length: 459
<
* Connection #0 to host localhost left intact
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}* Closing connection 0

I suggest you enable server side trace log with

PUT _cluster/settings
{
  "transient": {
    "logger.org.elasticsearch.xpack.security.authc": "trace"
  }
}

Capture and share the logs during authentication failure. Those logs should help diagnose it further.

Below is the verbose output of curl when trying to authenticate

curl https://localhost:9200/_cluster/health\?pretty --key config/certs/ca.key --cert config/certs/ca.crt --cacert config/certs/ca.crt -vv
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: config/certs/ca.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=node01
*  start date: Nov 10 01:17:25 2022 GMT
*  expire date: Nov  9 01:17:25 2025 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
> GET /_cluster/health?pretty HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Bearer realm="security"
< WWW-Authenticate: ApiKey
< WWW-Authenticate: Basic realm="security" charset="UTF-8"
< content-type: application/json; charset=UTF-8
< content-length: 718
<
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "missing authentication credentials for REST request [/_cluster/health?pretty]",
        "header" : {
          "WWW-Authenticate" : [
            "Bearer realm=\"security\"",
            "ApiKey",
            "Basic realm=\"security\" charset=\"UTF-8\""
          ]
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "missing authentication credentials for REST request [/_cluster/health?pretty]",
    "header" : {
      "WWW-Authenticate" : [
        "Bearer realm=\"security\"",
        "ApiKey",
        "Basic realm=\"security\" charset=\"UTF-8\""
      ]
    }
  },
  "status" : 401
}
* Connection #0 to host localhost left intact
* Closing connection 0

This is what is logged ( after enabling logger.org.elasticsearch.xpack.security.authc )

[2022-12-13T13:16:10,910][DEBUG][o.e.x.s.a.AuthenticationService] [node01] No valid credentials found in request [rest request uri [/_xpack/security/_authenticate?pretty]], rejecting

From that log entry, it appears Elasticsearch doesn't consider the certificate as a mode of authentication and is still looking for basic auth ( username and password ) which works as seen below

curl https://localhost:9200/_cluster/health\?pretty \
--key config/certs/ca.key --cert config/certs/ca.crt --cacert config/certs/ca.crt -vv -u user:password
*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: config/certs/ca.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=node01
*  start date: Nov 10 01:17:25 2022 GMT
*  expire date: Nov  9 01:17:25 2025 GMT
*  subjectAltName: host "localhost" matched cert's "localhost"
*  issuer: CN=Elastic Certificate Tool Autogenerated CA
*  SSL certificate verify ok.
* Server auth using Basic with user 'xxxx'
> GET /_cluster/health?pretty HTTP/1.1
> Host: localhost:9200
> Authorization: Basic c2FtaGFnaW46MTIzNDU2Nzg=
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 466
<
{
  "cluster_name" : "elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 1,
  "active_shards" : 1,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
* Connection #0 to host localhost left intact
* Closing connection 0

What must be done for Elasticsearch to only authenticate using the certificate? Thanks

What is the output of the get license API?

As Yang pointed out earlier, make sure you are using either a Platinum or Trial license.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.