I'm testing the Elasticsearch 7.2, thinking in upgrade my production cluster from 6.4 version to 7.2.
On my 6.4 cluster, I'm using SearchGuard for security, and works very well. I want to use X-Pack in the new version, but I can't configure HTTPS on Elastic's HTTP layer. I'm using my organization's certificate in a PEM format, the same I'm using on production's SearchGuard and Kibana currently. But on X-Pack I ever get the follow error: CertificateParsingException: signed fields invalid.
Please share the entire log message and not only part of it . Where do you get this ?
I can only guess that the file you have named as certificate.pem is not actually a PEM encoded X509 certificate, but something else ( maybe a DER encoded certificate or a keystore of some sort?)
Can you share the output of
openssl x509 -in certificate.pem -text -inform der
Sorry not share the log before. I have the limit of 7000 characters and I can't upload files except images, so I must remove log parts.
Message log
[2019-07-05T20:40:48,632][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-staging-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-7.2.0.jar:7.2.0]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-7.2.0.jar:7.2.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:115) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) ~[elasticsearch-7.2.0.jar:7.2.0]
Caused by: java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:614) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.node.Node.<init>(Node.java:307) ~[elasticsearch-7.2.0.jar:7.2.0]
... 6 more
Caused by: java.lang.reflect.InvocationTargetException
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
... 6 more
Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory
at org.elasticsearch.xpack.core.ssl.PEMTrustConfig.createTrustManager(PEMTrustConfig.java:48) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:382) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1133) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$2(SSLService.java:426) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1333) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:423) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:119) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.<init>(XPackPlugin.java:143) ~[?:?]
... 6 more
Caused by: java.security.cert.CertificateParsingException: signed fields invalid
at sun.security.x509.X509CertImpl.parse(X509CertImpl.java:1830) ~[?:?]
at sun.security.x509.X509CertImpl.<init>(X509CertImpl.java:188) ~[?:?]
at sun.security.provider.X509Factory.parseX509orPKCS7Cert(X509Factory.java:476) ~[?:?]
at sun.security.provider.X509Factory.engineGenerateCertificates(X509Factory.java:361) ~[?:?]
at java.security.cert.CertificateFactory.generateCertificates(CertificateFactory.java:478) ~[?:?]
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.readCertificates(CertParsingUtils.java:94) ~[?:?]
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.readCertificates(CertParsingUtils.java:86) ~[?:?]
at org.elasticsearch.xpack.core.ssl.PEMTrustConfig.createTrustManager(PEMTrustConfig.java:45) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:382) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1133) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$2(SSLService.java:426) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1333) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:423) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:119) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.<init>(XPackPlugin.java:143) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Constructor.java:481) ~[?:?]
at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.node.Node.<init>(Node.java:307) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.node.Node.<init>(Node.java:251) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:221) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:221) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:349) ~[elasticsearch-7.2.0.jar:7.2.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-7.2.0.jar:7.2.0]
... 6 more
No, not really at this point. This looks like the certificate you are using is malformed. Elasticsearch fails to start because it deems the configuration invalid, which in turn is caused by:
Caused by: java.security.cert.CertificateParsingException: signed fields invalid at
sun.security.x509.X509CertImpl.parse(X509CertImpl.java:1830) ~[?:?] at
sun.security.x509.X509CertImpl.<init>(X509CertImpl.java:188) ~[?:?] at
sun.security.provider.X509Factory.parseX509orPKCS7Cert(X509Factory.java:476) ~[?:?] at
sun.security.provider.X509Factory.engineGenerateCertificates(X509Factory.java:361) ~[?:?] at
java.security.cert.CertificateFactory.generateCertificates(CertificateFactory.java:478) ~[?:?]
[root@elasticsearch-gcp-staging bin]# /usr/share/elasticsearch/jdk/bin/java -version
openjdk version "12.0.1" 2019-04-16
OpenJDK Runtime Environment (build 12.0.1+12)
OpenJDK 64-Bit Server VM (build 12.0.1+12, mixed mode, sharing)
I'm using the same certificate on many other places, as the Elasticsearch using SearchGuard and are working fine.
From the link you posted above, the Exception are throw from the method private void parse(DerValue val), my certificate are not a DER, but a PEM certificate as you can see on previous posts. Has any other configurations I must add to my elasticsearch.yml file to force then to parse the certificate as a PEM format?
PEM formatted certificates contain DER encoded data so code eventually needs to parse the DER structure. See this awesome reply from Tim regarding certificate encoding, formatting, etc.
I'm not arguing against that, I'm only saying what your logs are implying.
What Java version are you using in your other Elasticsearch installation?
Can you try and run this Elasticsearch instance with Java 11 ?
Can you try and run your other Elasticsearch installation with Java 12 ?
Not sure what else to suggest without being able to test against your certificate/key.
Can you please try and create two files by copy-pasting
a) the Leaf certificate from the chain to an certificate.pem file and use that for xpack.security.http.ssl.certificate:
b) The CA certificate from the chain to a ca.pem file and use that for xpack.security.http.ssl.certificate_authorities:
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.