Elasticsearch drives me nuts when it comes to certificates and how they are used / configured.
This is my current configuration:
## Cluster Settings
cluster.name: "elk-tls-cluster"
node.name: node-1
network.host: "0.0.0.0"
http.host: 0.0.0.0
## License
xpack.license.self_generated.type: basic
# Security
xpack.security.enabled: true
xpack.security.authc.token.enabled: true
xpack.security.authc.api_key.enabled: true
xpack.security.enrollment.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
# transport security settings
# This is mostly used for inter-node communications between parts of the ELK stack
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.keystore.path: ${CONFIG_DIR}/elasticsearch.p12
xpack.security.transport.ssl.truststore.path: ${CONFIG_DIR}/elasticsearch_ca.p12
# HTTP security setttings
# This is used for client server ssl/tls communications (e.g. browser to kibana)
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: ${CONFIG_DIR}/elasticsearch.p12
xpack.security.http.ssl.truststore.path: ${CONFIG_DIR}/elasticsearch_ca.p12
"elasticsearch.p12" is the keystore containing my http certificates:
[elasticsearch]# keytool -list -v -keystore elasticsearch.p12
Enter keystore password:
***************** WARNING WARNING WARNING *****************
* The integrity of the information stored in your keystore *
* has NOT been verified! In order to verify its integrity, *
* you must provide your keystore password. *
***************** WARNING WARNING WARNING *****************
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: http
Creation date: Dec 12, 2023
Entry type: PrivateKeyEntry
*******************************************
*******************************************
"elasticsearch_ca.p12" is a keystore where I imported the root certificate (selfsigned root CA):
[elasticsearch]# keytool -list -v -keystore elasticsearch_ca.p12
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: root
Creation date: Dec 12, 2023
Entry type: trustedCertEntry
Owner: CN=Elastic Certificate Tool Autogenerated CA
Issuer: CN=Elastic Certificate Tool Autogenerated CA
Serial number: a2aadf5f4b5b675076f66712155e3614aeb82191
Valid from: Tue Feb 14 15:18:04 CET 2023 until: Fri Feb 13 15:18:04 CET 2026
Certificate fingerprints:
MD5: 87:8D:E0:F7:F3:38:BB:EF:07:6B:5C:60:3C:D2:F9:07
SHA1: 88:FC:2D:F9:19:3E:57:1F:74:73:5D:E7:01:11:7C:14:B5:BD:6C:98
SHA256: 02:8B:E9:90:25:82:AD:8B:47:60:21:FB:23:6B:C8:5E:72:DE:B5:DD:86:85:3A:7A:C9:3E:89:44:09:E1:29:57
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Extensions:
#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: BA 29 BD F2 F4 7A E2 1A 28 C4 F8 6B 18 FD D8 83 .)...z..(..k....
0010: F0 31 73 7C .1s.
]
]
#2: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
CA:true
PathLen:2147483647
]
#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: BA 29 BD F2 F4 7A E2 1A 28 C4 F8 6B 18 FD D8 83 .)...z..(..k....
0010: F0 31 73 7C .1s.
]
]
*******************************************
*******************************************
My problem is that I cannot use elasticsearch command line tools like "elasticsearch-reset-password" or "elasticsearch-create-enrollment-token"
I always the the following error message:
20:26:57.289 [main] WARN org.elasticsearch.common.ssl.DiagnosticTrustManager - failed to establish trust with server at [elasticsearch]; the server provided a certificate with subject name [CN=elasticsearch], fingerprint [fc50d2fa054952318e2b57c601001a1b6f81d32b], no keyUsage and no extendedKeyUsage; the certificate is valid between [2023-02-14T14:18:07Z] and [2026-02-13T14:18:07Z] (current time is [2023-12-12T20:26:57.286086601Z], certificate dates are valid); the session uses cipher suite [TLS_AES_256_GCM_SHA384] and protocol [TLSv1.3]; the certificate has subject alternative names [DNS:elasticsearch.fire.fly,DNS:localhost,IP:127.0.0.1,IP:0.0.0.0,DNS:elasticsearch]; the certificate is issued by [CN=Elastic Certificate Tool Autogenerated CA]; the certificate is signed by (subject [CN=Elastic Certificate Tool Autogenerated CA] fingerprint [52e7f3cffa59d6e4dc8f0a47c1d1c08560265ede]) which is self-issued; the [CN=Elastic Certificate Tool Autogenerated CA] certificate is not trusted in this ssl context ([xpack.security.http.ssl (with trust configuration: StoreTrustConfig{path=/usr/share/elasticsearch/config/elasticsearch_ca.p12, password=<non-empty>, type=PKCS12, algorithm=PKIX})]); this ssl context does trust a certificate with subject [CN=Elastic Certificate Tool Autogenerated CA] but the trusted certificate has fingerprint [88fc2df9193e571f74735de701117c14b5bd6c98]
I guess the truststore needs to be configured on some other locations - but wondering where.
Any ideas?