Elasticsearch-certutil add node pem format failing

In prep for upgrade to 7x from 6.61, I am creating new certs. After creating the initial certs without issue, I attempted to add a node and it is failing using the following command:
bin/elasticsearch-certutil cert --pem --ca ca/ca.crt --multiple

Please see output below for the method.

NOTE: The "initially created" certs are working fine.
Thanks for any assistance.

NOTE2: I did not create a password for ca, in other words, I hit the "enter" at the password prompt.

bin/elasticsearch-certutil cert ca --pem --in nodes.yml --out ent-certs.zip
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.bouncycastle.jcajce.provider.drbg.DRBG (file:/usr/share/elasticsearch/lib/tools/security-cli/bcprov-jdk15on-1.61.jar) to constructor sun.security.provider.Sun()
WARNING: Please consider reporting this to the maintainers of org.bouncycastle.jcajce.provider.drbg.DRBG
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

---DEFAULT TOOL MESSAGES OMITTED FOR SPACE----

Certificates written to /usr/share/elasticsearch/ent-certs.zip

This file should be properly secured as it contains the private keys for
all instances.

After unzipping the file, there will be a directory for each instance.
Each instance has a certificate and private key.
For each Elastic product that you wish to configure, you should copy
the certificate, key, and CA certificate to the relevant configuration directory
and then follow the SSL configuration instructions in the product guide.

For client applications, you may only need to copy the CA certificate and
configure the client to trust this certificate.

unzip ent-certs.zip
Archive: ent-certs.zip
creating: ca/
inflating: ca/ca.crt
creating: entelkmst-ch2-01/
inflating: entelkmst-ch2-01/entelkmst-ch2-01.crt
inflating: entelkmst-ch2-01/entelkmst-ch2-01.key
creating: entelkmst-ch2-02/
inflating: entelkmst-ch2-02/entelkmst-ch2-02.crt
inflating: entelkmst-ch2-02/entelkmst-ch2-02.key
creating: entelkmst-ch2-03/
inflating: entelkmst-ch2-03/entelkmst-ch2-03.crt
inflating: entelkmst-ch2-03/entelkmst-ch2-03.key
creating: wdv-log17/
inflating: wdv-log17/wdv-log17.crt
inflating: wdv-log17/wdv-log17.key
creating: wdv-log18/
inflating: wdv-log18/wdv-log18.crt
inflating: wdv-log18/wdv-log18.key
creating: elkdata-cmc-01/
inflating: elkdata-cmc-01/elkdata-cmc-01
inflating: elkdata-cmc-01/elkdata-cmc-01key

 bin/elasticsearch-certutil cert --pem --ca ca/ca.crt --multiple
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.bouncycastle.jcajce.provider.drbg.DRBG (file:/usr/share/elasticsearch/lib/tools/security-cli/bcprov-jdk15on-1.61.jar) to constructor sun.security.provider.Sun()
WARNING: Please consider reporting this to the maintainers of org.bouncycastle.jcajce.provider.drbg.DRBG
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
This tool assists you in the generation of X.509 certificates and certificate
signing requests for use with SSL/TLS in the Elastic stack.

---****DEFAULT TOOL MESSAGES OMITTED FOR SPACE****----

Enter password for CA (ca/ca.crt) :
Exception in thread "main" java.io.IOException: toDerInputStream rejects tag type 45
        at java.base/sun.security.util.DerValue.toDerInputStream(DerValue.java:873)
        at java.base/sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:1981)
        at java.base/sun.security.util.KeyStoreDelegator.engineLoad(KeyStoreDelegator.java:222)
        at java.base/java.security.KeyStore.load(KeyStore.java:1472)
        at org.elasticsearch.xpack.core.ssl.CertParsingUtils.readKeyStore(CertParsingUtils.java:71)
        at org.elasticsearch.xpack.core.ssl.CertParsingUtils.readPkcs12KeyPairs(CertParsingUtils.java:130)
        at org.elasticsearch.xpack.security.cli.CertificateTool$CertificateCommand.lambda$loadPkcs12CA$1(CertificateTool.java:341)
        at org.elasticsearch.xpack.security.cli.CertificateTool.withPassword(CertificateTool.java:931)
        at org.elasticsearch.xpack.security.cli.CertificateTool.access$100(CertificateTool.java:85)
        at org.elasticsearch.xpack.security.cli.CertificateTool$CertificateCommand.loadPkcs12CA(CertificateTool.java:340)
        at org.elasticsearch.xpack.security.cli.CertificateTool$CertificateCommand.getCAInfo(CertificateTool.java:328)
        at org.elasticsearch.xpack.security.cli.CertificateTool$GenerateCertificateCommand.execute(CertificateTool.java:684)
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.MultiCommand.execute(MultiCommand.java:77)
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
        at org.elasticsearch.cli.Command.main(Command.java:90)
        at org.elasticsearch.xpack.security.cli.CertificateTool.main(CertificateTool.java:137)

--ca expects a PKCS#12 store and you are passing a PEM encoded certifcate, so it fails. See https://www.elastic.co/guide/en/elasticsearch/reference/current/certutil.html

--ca <file_path>
Specifies the path to an existing CA key pair (in PKCS#12 format). This parameter cannot be used with the ca or csr parameters.

You should use --ca-cert and --ca-key instead to pass the CA key and certificate since you have them in PEM format.

An issue is that when you created the CA key and certificate in the first command with

bin/elasticsearch-certutil cert ca --pem

you didn't specify --keep-ca-key so the key was not stored on file. That's unfortunate because you only have the CA cert now and you cannot sign any more certificates with that CA. You probably need to generate a new CA key and certificate and regenerate your node certificates.

1 Like

Thanks very much for the response. I am glad to know this for moving forward.