Curl against an encrypted Elasticsearch instance with certificate verification

Elastic Stack Elasticsearch
1

Hi there!

SHORT QUESTION:
How can I make a curl request to a local ES cluster (3 nodes running on localhost) in the form of curl -XGET "https://localhost:9200" -u elastic --cacert elasticsearch-ca.pem, verifying the certificates and without getting as response something like:

curl: (60) SSL: no alternative certificate subject name matches target host name '127.0.0.1' ?

LONG VERSION:
I set up a cluster of 3 nodes, each having the following elasticsearch.yml file:

cluster.name: ssl_enabled_test_cluster
node.name: node-1
cluster.initial_master_nodes: ["node-1", "node-2", "node-3"]
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: certs/elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: certs/elastic-stack-ca.p12
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: certs/http.p12

(obviously node-2 and node-3 for the other 2 nodes)

I generated the certificates for the transport layer (elastic-certificates.p12 and elastic-stack-ca.p12) with:

bin/elasticsearch-certutil ca <-- elastic-stack-ca.p12
bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 <-- elastic-certificates.p12

I generated those for http (http.p12 for elasticsearch and elasticsearch-ca.pem for Kibana) with:

bin/elasticsearch-certutil http

Then, I put elastic-stack-ca.p12, elastic-certificates.p12 and http.p12 in config/certs folder of each node and the elasticsearch-ca.pem file in the config/certs folder of kibana, whose kibana.yml file is the following (fyi):

elasticsearch.hosts: ["https://localhost:9200"]
elasticsearch.ssl.certificateAuthorities: [ "config/certs/elasticsearch-ca.pem" ]
elasticsearch.ssl.verificationMode: certificate
elasticsearch.username: "kibana_system"
elasticsearch.password: "elastic"

Now, elasticsearch cluster works as expected, starting and running without any problem and a classic curl without certificate verification (i.e. curl -XGET "https://localhost:9200" -u elastic -k) returns a successful message. Kibana has no problem connecting to the cluster since the verificationMode is set to certificate, so the hostname verification is skipped.

Obviously, if I run a curl without the -k flag, it returns the error

SSL certificate problem: self signed certificate in certificate chain

since the certificate is a self-signed one. Then I thought I just needed to pass the same CA generated by the bin/elasticsearch-certutil http and used by kibana to connect to elasticsearch through ssl. So I went with

curl -XGET "https://localhost:9200" -u elastic --cert elasticsearch-ca.pem

which basically returns the error cause of this post:

SSL: no alternative certificate subject name matches target host name 'localhost'

Same obviously happens to Kibana if I run it with with elasticsearch.ssl.verificationMode: full, which doesn't skip the hostname verification, returning the error:

Hostname/IP does not match certificate's altnames: Host: localhost. is not in the cert's altnames: DNS:node-3, DNS:node-2, DNS:node-1

Now, here's what I put when I run the bin/elasticsearch-certutil http:

Generate a CSR? [y/N] N
Use an existing CA? [y/N] y
CA Path: ....elastic-stack-ca.p12
Password for elastic-stack-ca.p12: *****
For how long should your certificate be valid? [5y]: 10y
Generate a certificate per node? [y/N]: N
Enter all the hostnames that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.

localhost
_local_
node-1
node-2
node-3

You entered the following hostnames.

 - localhost
 - _local_
 - node-1
 - node-2
 - node-3

Is this correct [Y/n]:Y 

Enter all the IP addresses that you need, one per line.
When you are done, press <ENTER> once more to move on to the next step.

127.0.0.1

You entered the following IP addresses.

 - 127.0.0.1

Is this correct [Y/n]Y

## Other certificate options

The generated certificate will have the following additional configuration
values. These values have been selected based on a combination of the
information you have provided above and secure defaults. You should not need to
change these values unless you have specific requirements.

Key Name: localhost
Subject DN: CN=localhost
Key Size: 2048

Do you wish to change any of these options? [y/N]:N

Provide a password for the "http.p12" file: ****
Repeat password to confirm: ****

What filename should be used for the output zip file? [...elasticsearch-ssl-http.zip]

I reported here everything since I'm quite sure the problem is in how I generated the http.12-elasticsearch-ca.pem tuple.

Any ideas? Thanks!

1 Like
2

Ok thanks to a colleague of mine I finally found how to make it work for a localhost instance (or cluster as in my case). Here are the steps:

  1. Create a file instance.yml like the following:

    instances:

    • name: 'node'
      dns: ['localhost']
      ip: ['127.0.0.1']

  2. Create the certificate with the following command of the certutil tool:

bin/elasticsearch-certutil cert --keep-ca-key ca --pem --in instance.yml --out certs.zip

It will spit out a zip file with the following structure (once extracted in the certs folder):

certs
├── ca
│   ├── ca.crt
│   └── ca.key
└── node
    ├── node.crt
    └── node.key

  1. Create a folder (e.g. certs) at the path $ES_HOME/config and copy the files ca.crt, node.crt and node.key in that folder.

  2. Configure the ssl part in the elasticsearch.yml file as following:

    xpack.security.enabled: true
    xpack.security.transport.ssl.enabled: true
    xpack.security.http.ssl.enabled: true
    xpack.security.http.ssl.key: certs/node.key
    xpack.security.http.ssl.certificate: certs/node.crt
    xpack.security.http.ssl.certificate_authorities: ["certs/ca.crt"]
    xpack.security.transport.ssl.key: certs/node.key
    xpack.security.transport.ssl.certificate: certs/node.crt
    xpack.security.transport.ssl.certificate_authorities: ["certs/ca.crt"]

  3. Start Elasticsearch

  4. Set up authentication with

bin/elasticsearch-setup-passwords interactive

  1. Now you can cURL elasticsearch without ignoring the certificate with

curl --cacert $ES_HOME/config/certs/ca.crt -XGET "https://localhost:9200" -u elastic

  1. If you want to have a Kibana instance connecting to the ES instance with full verification mode (without skipping the hostname verification) you can create a folder in the $KIBANA_HOME/config folder (e.g. certs) and copy there the ca.crt certificate file. Then, in your $KIBANA_HOME/config/kibana.yml file you set:

    elasticsearch.hosts: ["https://localhost:9200"]
    elasticsearch.ssl.certificateAuthorities: [ "config/certs/ca.crt" ]
    elasticsearch.ssl.verificationMode: full
    elasticsearch.username: "kibana_system"
    elasticsearch.password: "pwd-you-set-for-kibana_system-user"

P.S. note that full is the default value for elasticsearch.ssl.verificationMode, so no need to specify it.

Hope it helps anybody trying to set a local instance with certificates to carry out tests like me.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.