# Configure PKI authentication for Elasticsearch 7.7

**URL:** <https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [November 10, 2022, 7:11pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686 "2022-11-10T19:11:15Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![samhagin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samhagin/32/77283_2.png) [@samhagin](https://discuss.elastic.co/u/samhagin)\
**Post date:** [November 10, 2022, 7:11pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/1 "2022-11-10T19:11:15Z")

</div>

This guide [Elasticsearch Security: Configure TLS/SSL & PKI Authentication | Elastic Blog](https://www.elastic.co/blog/elasticsearch-security-configure-tls-ssl-pki-authentication) shows how to configure PKI for Elasticsearch version 6.5 however this doesn't work for version 7.7.0 ( the version I'm currently using ). When I add `xpack.security.authc.realms.pki1.type: pki` to my elasticsearch.yml file, I get the error below:

`java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.security.Security] Likely root cause: java.lang.IllegalArgumentException: Incorrect realm settings found. Realm settings have been changed to include the type as part of the setting key. For example 'xpack.security.authc.realms.file.my_file.order' Found invalid config: xpack.security.authc.realms.pki1.type Please see the breaking changes documentation. `

I went through the documentation for Elasticsearch 7.7 but didn't find anything specific to PKI changes. Can someone please assist with the settings to use and also how to generate the client certificate for authentication? Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 10, 2022, 7:11pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/2 "2022-11-10T19:11:15Z")

</div>

Elasticsearch 7.7 is [EOL](https://www.elastic.co/support/eol) and no longer supported. Please upgrade ASAP.

(This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: )

---

<div class="post-metadata">

**Author:** ![Justin\_Cranford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/justin_cranford/32/85302_2.png) [@Justin\_Cranford](https://discuss.elastic.co/u/Justin_Cranford)\
**Post date:** [November 10, 2022, 10:26pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/3 "2022-11-10T22:26:17Z")

</div>

Does this help?

> **[PKI user authentication | Elasticsearch Guide \[7.7\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.7/pki-realm.html)**

> Add a realm configuration for a `pki` realm to `elasticsearch.yml` under the `xpack.security.authc.realms.pki` namespace.

---

<div class="post-metadata">

**Author:** ![samhagin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samhagin/32/77283_2.png) [@samhagin](https://discuss.elastic.co/u/samhagin)\
**Post date:** [November 11, 2022, 5:08pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/4 "2022-11-11T17:08:01Z")

</div>

Thanks for the reply. Using the settings below works. That is Elasticsearch now starts up without an error. Would you know how I can configure the client and server certificates to use for mutual authentication?

```auto
xpack:
  security:
    authc:
      realms:
        pki:
          pki1:
            order: 1

```

---

<div class="post-metadata">

**Author:** ![samhagin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samhagin/32/77283_2.png) [@samhagin](https://discuss.elastic.co/u/samhagin)\
**Post date:** [November 15, 2022, 12:36pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/5 "2022-11-15T12:36:28Z")

</div>

After further research, I was able to use the certutil command to generate a client cert. Elasticsearch however still seems to require a username and password.

Command

```auto
bin/elasticsearch-certutil cert --ca-cert certs/ca/ca.crt --ca-key certs/ca/ca.key -out client.p12

```

I then used this client.p12 cert to authenticate to the cluster

```auto
curl -k --cert-type P12 --cert client.p.12 https://localhost:9200

```

I get a 401 error

```auto
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}%

```

I'm able to authenticate when I provide the username and password

```auto
 curl -k --cert-type P12 --cert client.p.12 https://localhost:9200 -u user:password
{
  "name" : "node01",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "W0MJXDsnTw2LlnhGHw3Lug",
  "version" : {
    "number" : "7.7.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "81a1e9eda8e6183f5237786246f6dced26a10eaf",
    "build_date" : "2020-05-12T02:01:37.602180Z",
    "build_snapshot" : false,
    "lucene_version" : "8.5.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

```

How do I connect to Elasticsearch with the certificate only and NOT provide a username and password? My elasticsearch.yml is below

```auto
node.name: node01
xpack.security.enabled: true
discovery.type: single-node
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/node01.key
xpack.security.transport.ssl.certificate: certs/node01.crt
xpack.security.transport.ssl.certificate_authorities: ["certs/ca.crt"]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/node01.key
xpack.security.http.ssl.certificate: certs/node01.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.security.http.ssl.client_authentication: optional

```

Thanks for your help.

---

<div class="post-metadata">

**Author:** ![samhagin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samhagin/32/77283_2.png) [@samhagin](https://discuss.elastic.co/u/samhagin)\
**Post date:** [December 5, 2022, 6:04pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/6 "2022-12-05T18:04:05Z")

</div>

I've still not found a solution to Elasticsearch still requiring basic auth despite trying to use a certificate to authenticate. What setting(s) do I have to add to elasticsearch.yml to make cert authentication work?

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [December 5, 2022, 11:13pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/7 "2022-12-05T23:13:31Z")

</div>

What is the complete content of your `elasticsearch.yml` file? The one you shared immediately above does not include the PKI realm settings.

PKI realm is a license feature. So you'll need either a platinum license or trial license. Otherwise, the PKI realm will be skipped.

---

<div class="post-metadata">

**Author:** ![samhagin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samhagin/32/77283_2.png) [@samhagin](https://discuss.elastic.co/u/samhagin)\
**Post date:** [December 6, 2022, 5:59am UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/8 "2022-12-06T05:59:04Z")

</div>

This is the entire elasticsearch.yml file

```auto
node.name: node01
xpack.security.enabled: true
discovery.type: single-node
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.key: certs/node01.key
xpack.security.transport.ssl.certificate: certs/node01.crt
xpack.security.transport.ssl.certificate_authorities: ["certs/ca.crt"]
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.key: certs/node01.key
xpack.security.http.ssl.certificate: certs/node01.crt
xpack.security.http.ssl.certificate_authorities: certs/ca.crt
xpack.security.http.ssl.client_authentication: required
xpack.security.http.ssl.verification_mode: certificate

xpack:
  security:
    authc:
      realms:
        pki:
          pki1:
            order: 1
            certificate_authorities: ["certs/ca.crt"]

```

when I try to connect without the -k flag I get the error below

```auto
curl --cert-type P12 --cert client_node.p12 https://localhost:9200

curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

```

and in verbose mode

```auto
curl --cert-type P12 --cert client_node.p12 https://localhost:9200 -vvv
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
* SSL certificate problem: unable to get local issuer certificate
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

```

which certificate is curl trying to verify and is the command I'm using the right way to connect using SSL authentication?

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [December 7, 2022, 6:35am UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/9 "2022-12-07T06:35:56Z")

</div>

The error means the client (curl) does not trust the server's certificate. It has nothing to do with PKI realm. In the verbose log it says

> CAfile: /etc/ssl/cert.pem

I don't think your ES server cert is in that location. So you might want to consider specify `--cacert` option for curl.

---

<div class="post-metadata">

**Author:** ![samhagin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samhagin/32/77283_2.png) [@samhagin](https://discuss.elastic.co/u/samhagin)\
**Post date:** [December 7, 2022, 2:48pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/10 "2022-12-07T14:48:26Z")

</div>

Thanks for your help so far. So I added the `--cacert` option and I was able to get past the `unable to get local issuer certificate` error but curl still wants to use basic auth. Is there a setting or a way to ensure only certificate authentication works and basic auth is disabled?

```auto
curl --cert-type P12 --cert client_node.p12 https://localhost:9200 --cacert config/certs/ca.crt -vvv
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: config/certs/ca.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=node01
* start date: Nov 10 01:17:25 2022 GMT
* expire date: Nov 9 01:17:25 2025 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
> GET / HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Bearer realm="security"
< WWW-Authenticate: ApiKey
< WWW-Authenticate: Basic realm="security" charset="UTF-8"
< content-type: application/json; charset=UTF-8
< content-length: 459
<
* Connection #0 to host localhost left intact
{"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/]","header":{"WWW-Authenticate":["Bearer realm=\"security\"","ApiKey","Basic realm=\"security\" charset=\"UTF-8\""]}},"status":401}* Closing connection 0

```

---

<div class="post-metadata">

**Author:** ![Yang\_Wang](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yang_wang/32/48857_2.png) [@Yang\_Wang](https://discuss.elastic.co/u/Yang_Wang)\
**Post date:** [December 13, 2022, 5:49am UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/11 "2022-12-13T05:49:51Z")

</div>

I suggest you enable server side trace log with

```auto
PUT _cluster/settings
{
  "transient": {
    "logger.org.elasticsearch.xpack.security.authc": "trace"
  }
}

```

Capture and share the logs during authentication failure. Those logs should help diagnose it further.

---

<div class="post-metadata">

**Author:** ![samhagin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samhagin/32/77283_2.png) [@samhagin](https://discuss.elastic.co/u/samhagin)\
**Post date:** [December 13, 2022, 8:40pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/12 "2022-12-13T20:40:28Z")

</div>

Below is the verbose output of curl when trying to authenticate

```auto
curl https://localhost:9200/_cluster/health\?pretty --key config/certs/ca.key --cert config/certs/ca.crt --cacert config/certs/ca.crt -vv

```

```auto
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: config/certs/ca.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=node01
* start date: Nov 10 01:17:25 2022 GMT
* expire date: Nov 9 01:17:25 2025 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
> GET /_cluster/health?pretty HTTP/1.1
> Host: localhost:9200
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< WWW-Authenticate: Bearer realm="security"
< WWW-Authenticate: ApiKey
< WWW-Authenticate: Basic realm="security" charset="UTF-8"
< content-type: application/json; charset=UTF-8
< content-length: 718
<
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "missing authentication credentials for REST request [/_cluster/health?pretty]",
        "header" : {
          "WWW-Authenticate" : [
            "Bearer realm=\"security\"",
            "ApiKey",
            "Basic realm=\"security\" charset=\"UTF-8\""
          ]
        }
      }
    ],
    "type" : "security_exception",
    "reason" : "missing authentication credentials for REST request [/_cluster/health?pretty]",
    "header" : {
      "WWW-Authenticate" : [
        "Bearer realm=\"security\"",
        "ApiKey",
        "Basic realm=\"security\" charset=\"UTF-8\""
      ]
    }
  },
  "status" : 401
}
* Connection #0 to host localhost left intact
* Closing connection 0

```

This is what is logged ( after enabling `logger.org.elasticsearch.xpack.security.authc` )

```auto
[2022-12-13T13:16:10,910][DEBUG][o.e.x.s.a.AuthenticationService] [node01] No valid credentials found in request [rest request uri [/_xpack/security/_authenticate?pretty]], rejecting

```

From that log entry, it appears Elasticsearch doesn't consider the certificate as a mode of authentication and is still looking for basic auth ( username and password ) which works as seen below

```auto
curl https://localhost:9200/_cluster/health\?pretty \
--key config/certs/ca.key --cert config/certs/ca.crt --cacert config/certs/ca.crt -vv -u user:password
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 9200 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: config/certs/ca.crt
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: CN=node01
* start date: Nov 10 01:17:25 2022 GMT
* expire date: Nov 9 01:17:25 2025 GMT
* subjectAltName: host "localhost" matched cert's "localhost"
* issuer: CN=Elastic Certificate Tool Autogenerated CA
* SSL certificate verify ok.
* Server auth using Basic with user 'xxxx'
> GET /_cluster/health?pretty HTTP/1.1
> Host: localhost:9200
> Authorization: Basic c2FtaGFnaW46MTIzNDU2Nzg=
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< content-type: application/json; charset=UTF-8
< content-length: 466
<
{
  "cluster_name" : "elasticsearch",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 1,
  "number_of_data_nodes" : 1,
  "active_primary_shards" : 1,
  "active_shards" : 1,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
* Connection #0 to host localhost left intact
* Closing connection 0

```

What must be done for Elasticsearch to only authenticate using the certificate? Thanks

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [December 13, 2022, 8:45pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/13 "2022-12-13T20:45:56Z")

</div>

What is the output of the [get license API](https://www.elastic.co/guide/en/elasticsearch/reference/7.8/get-license.html)?

> [@Yang\_Wang](#):
>
> PKI realm is a license feature. So you'll need either a platinum license or trial license. Otherwise, the PKI realm will be skipped.

As Yang pointed out earlier, make sure you are using either a Platinum or Trial license.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 10, 2023, 8:46pm UTC](https://discuss.elastic.co/t/configure-pki-authentication-for-elasticsearch-7-7/318686/14 "2023-01-10T20:46:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.