Elasticsearch-create-enrollment-token is not possible without a keystore ( aka with PEM certificates)

Elastic Stack Elasticsearch
3

Ah, Thanks ... I've been sys-admin'ing Elasticsearch for 4+ years, so I go back to 7.x. I'm setting up a brand new cluster for the first time in a year or two, 8.8.1 in a new QA environment. Previously bootstrapping the cluster didn't now work with this config, so I had gone back to the "Getting Starting ..." docs - the RPM installation ... is till a bit awakard, for setting up PEM Certificate clusters.

However, for the sake of the next person that google's this is how to boot-strap/create a new cluster - QAW1 (QA-West 1) - with a four node cluster, using Let's Encrypt wild-card certificate on both transport and http.

This is the elasticsearch.yml from es1.qaw1 , and started them all at the same time with ansible ... ansible -f 9 -u dpd [ ...] -a "systemctl start elasticsearch ":

cluster.name: qaw1
node.name: es1.qaw1
node.attr.site: qaw1
node.attr.host: es1


path.data: /elastic/data
path.logs: /var/log/elasticsearch
path:
  repo:
    - /z/elastic
    - /z/elastic/snapshots
network.host: 10.11.162.11
transport.host: 10.11.162.11
http.host: 10.11.162.11

discovery.seed_hosts:
    - es1.qaw1
    - es2.qaw1
    - es3.qaw1
    - es4.qaw1
cluster.initial_master_nodes:
    - es1.qaw1
    - es2.qaw1
    - es3.qaw1
    - es4.qaw1

xpack.security.enrollment.enabled: true
xpack.security.autoconfiguration.enabled: false
xpack.security.enabled: true


# Enable encryption and mutual authentication between cluster nodes
xpack.security.transport.ssl:
    enabled: true
    verification_mode: certificate
    key: /etc/elasticsearch/ssl/certs/corp/key.pem
    certificate: /etc/elasticsearch/ssl/certs/corp/chain.pem
    certificate_authorities: [ "/etc/elasticsearch/ssl/certs/corp/le-ca-RSA-chain.pem", "/etc/elasticsearch/ssl/certs/corp/le-ca.pem", "/etc/elasticsearch/ssl/certs/corp/ca-bundle-chain.pem" ]

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
xpack.security.http.ssl:
    enabled: true
    verification_mode: certificate
    key:  /etc/elasticsearch/ssl/certs/corp/key.pem
    certificate: /etc/elasticsearch/ssl/certs/corp/chain.pem
    certificate_authorities: [ "/etc/elasticsearch/ssl/certs/corp/le-ca.pem", "/etc/elasticsearch/ssl/certs/corp/le-ca-RSA-chain.pem", "/etc/elasticsearch/ssl/certs/corp/ca-bundle-chain.pem" ]

For those using Let's Encrypt specifically, but certificate management scripts - after the recent LE Root expiring - my CA bundles are built like this :

	#Intermediate Certificates

	R3="lets-encrypt-r3.pem"
	E1="lets-encrypt-e1.pem"

	# Root
	X2="isrg-root-x2.pem"
	X1="isrgrootx1.pem"
	X1XS="isrg-root-x1-cross-signed.pem"

	#backup
	R4="lets-encrypt-r4.pem"
	E2=""
	

    # le-ca.pem - full RSA chain.
    LE_CA_FILES=
    for c in $R3 $R4 $X2 $X1; do
        if [ -n "${LE_CA_FILES}" ]; then
            LE_CA_FILES="${LE_CA_FILES} "
        fi
        LE_CA_FILES="${LE_CA_FILES}${ACMEPATH_NFS}/le-root-certs/$c"
    done
    cat ${CABUNDLE} ${LE_CA_FILES} > ${CERTS}/${SRV}/ca-bundle-chain.pem.tmp
    cat ${LE_CA_FILES} > ${CERTS}/${SRV}/le-ca-RSA-chain.pem.tmp
    cat ${ACMEPATH_NFS}/le-root-certs/$R3 ${ACMEPATH_NFS}/le-root-certs/$X1 > ${CERTS}/${SRV}/le-ca.pem.tmp

    for capem in ca-bundle-chain.pem le-ca-RSA-chain.pem le-ca.pem; do
        diff ${CERTS}/${SRV}/$capem ${CERTS}/${SRV}/${capem}.tmp  > /dev/null
        if [ $? -eq 1 -o ! -f "${CERTS}/${SRV}/${capem}" ]; then
            mv -fv ${CERTS}/${SRV}/${capem}.tmp ${CERTS}/${SRV}/${capem}
        fi
    done

And ... ( note, with signed LE certificates, curl built in CAs will validate the certificates, but a FQDN, matching the CN= in the certificate ... needs to be used. )

curl -u elastic:*****  -X GET "https://es1.qaw1.${FQDN}:9200/_cluster/health?pretty"
{
  "cluster_name" : "qaw1",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 4,
  "number_of_data_nodes" : 4,
  "active_primary_shards" : 1,
  "active_shards" : 2,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}
2 Likes