Settings SSL/TLS with xpack

Elastic Stack Elasticsearch
1

Hello, I'm trying to set up SSL/TLS with x-pack and running into this on some nodes :

 [WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler]  uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin]
Caused by: java.lang.IllegalArgumentException: could not parse pem certificate
        at org.elasticsearch.xpack.core.ssl.CertUtils.readCertificates(CertUtils.java:330) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.CertUtils.readCertificates(CertUtils.java:315) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.CertUtils.readCertificates(CertUtils.java:307) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.PEMKeyConfig.getCertificateChain(PEMKeyConfig.java:86) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.PEMKeyConfig.createKeyManager(PEMKeyConfig.java:77) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:421) ~[?:?]
        at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_161]
        at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:471) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:91) ~[?:?]
        at org.elasticsearch.xpack.core.XPackPlugin.<init>(XPackPlugin.java:127) ~[?:?]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_161]
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:554) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:505) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:422) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:146) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.node.Node.<init>(Node.java:303) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-6.2.3.jar:6.2.3]
        ... 6 more

My configuration elasticsearch.yml

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.key: /etc/elasticsearch/cert/elastic.key
xpack.security.transport.ssl.certificate: /etc/elasticsearch/cert/elastic.crt
xpack.security.transport.ssl.certificate_authorities: [ "/etc/elasticsearch/cert/vaca.crt"]

please help to resolve above error ?

2

Hi

Is your /etc/elasticsearch/cert/elastic.crt PEM encoded ? How did you create it ?

Does the file begin with '-----BEGIN CERTIFICATE----- ' ?
Can you parse it with openssl x509 -in /etc/elasticsearch/cert/elastic.crt -text -noout ?

If the answer is no and you get an error similar to

unable to load certificate
12626:error:0906D06C:PEM routines:PEM_read_bio:no start line

, it could be that it's DER encoded and you need to modify it to PEM using openssl
openssl x509 -inform der -in elastic.crt -out elastic.pem and change the elasticsearch.yml accordingly.

Keep in mind the the .pem file suffix is not mandatory, you could name it however you want.

3

Hi
Thank you for your reply, unfortunately my certificate elastic.crt is really in PEM encoded format, begin with '------BEGIN CERTIFICATE-------'
and i can parse it with openssl using your command. Logs shows :

[ERROR][o.e.b.Bootstrap          ] Exception
java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin]
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:563) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:505) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:422) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:146) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.node.Node.<init>(Node.java:303) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.node.Node.<init>(Node.java:246) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:213) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.3.jar:6.2.3]
        at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.3.jar:6.2.3]
        at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.3.jar:6.2.3]
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_161]
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:554) ~[elasticsearch-6.2.3.jar:6.2.3]
        ... 15 more
Caused by: java.lang.IllegalArgumentException: could not parse pem certificate
        at org.elasticsearch.xpack.core.ssl.CertUtils.readCertificates(CertUtils.java:330) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.CertUtils.readCertificates(CertUtils.java:315) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.CertUtils.readCertificates(CertUtils.java:307) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.PEMKeyConfig.getCertificateChain(PEMKeyConfig.java:86) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.PEMKeyConfig.createKeyManager(PEMKeyConfig.java:77) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:421) ~[?:?]
        at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_161]
        at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:471) ~[?:?]
        at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:91) ~[?:?]
        at org.elasticsearch.xpack.core.XPackPlugin.<init>(XPackPlugin.java:127) ~[?:?]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_161]
        at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:554) ~[elasticsearch-6.2.3.jar:6.2.3]
        ... 15 more

Any idea ? thanks

4 
 openssl x509 -inform der -in elastic.crt -out elastic.pem
unable to load certificate
140005390264000:error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1199:
140005390264000:error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:374:Type=X509
5

Sorry for spamming but the weird thing is that one of my 3 elasticsearch node start successfully and don't show this Warn while the 2 others fails and cannot parse the same certificate.

6

Not really sure how this can happen. Could you please verify that this is the same certificate indeed in all three nodes so that we can eliminate a corrupted file during copying ?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.