I am tasked with trying to get Elasticsearch working in FIPS mode. I know there is probably no experts on this topic in the forums but wanted to see if anyone could give me just a working solution.
In my Elasticsearch.yml i have.
## FIPS
xpack.security.fips_mode.enabled: true
xpack.security.authc.password_hashing.algorithm: pbkdf2_stretch
# SSL Stuff
xpack.security.enabled: true
xpack.security.http.ssl.enabled: true
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.http.ssl.keystore.password: {12 character password}
xpack.security.http.ssl.truststore.password: {12 character password}
xpack.security.http.ssl.keystore.type: BCFKS
xpack.security.http.ssl.truststore.type: BCFKS
xpack.security.http.ssl.keystore.path: /etc/elasticsearch/node.bcfks
xpack.security.http.ssl.truststore.path: /etc/elasticsearch/node.bcfks
xpack.security.http.ssl.supported_protocols: TLSv1, TLSv1.1, TLSv1.2, TLSv1.3
I am using the embedded JAVA JVM included with Elasticsearch 7.15.1, and I modified the conf/security/java.security as follows:
security.provider.1=org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider C:DEFRND[HmacSHA512];ENABLE{ALL};
security.provider.2=org.bouncycastle.jsse.provider.BouncyCastleJsseProvider fips:BCFIPS crypto.policy=unlimited
security.provider.3=SUN
security.provider.4=SunRsaSign
security.provider.5=SunEC
security.provider.6=SunJSSE
security.provider.7=SunJCE
security.provider.8=SunJGSS
security.provider.9=SunSASL
security.provider.10=XMLDSig
security.provider.11=SunPCSC
security.provider.12=JdkLDAP
security.provider.13=JdkSASL
security.provider.14=SunPKCS11
keystore.type=BCFKS
keystore.type.compat=false
Nothing else was modified from the original java.security.
As of right now i get the JAVA error when trying to start Elasticsearch
[2022-04-26T14:09:38,281][ERROR][o.e.b.Bootstrap ] [node-1] Exception
org.elasticsearch.ElasticsearchSecurityException: failed to load SSL configuration [xpack.security.transport.ssl]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:530) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1420) ~[?:?]
at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1553) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:526) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:144) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:459) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:301) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$18(Node.java:622) ~[elasticsearch-7.15.1.jar:7.15.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.<init>(Node.java:626) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.node.Node.<init>(Node.java:288) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) [elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:167) [elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:158) [elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) [elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:114) [elasticsearch-cli-7.15.1.jar:7.15.1]
at org.elasticsearch.cli.Command.main(Command.java:79) [elasticsearch-cli-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:123) [elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) [elasticsearch-7.15.1.jar:7.15.1]
Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory
at org.elasticsearch.xpack.core.ssl.DefaultJDKTrustConfig.createTrustManager(DefaultJDKTrustConfig.java:51) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:439) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1219) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:528) ~[?:?]
... 26 more
Caused by: java.security.KeyStoreException: problem accessing trust store
at sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:73) ~[?:?]
at javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:282) ~[?:?]
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.trustManager(CertParsingUtils.java:297) ~[?:?]
at org.elasticsearch.xpack.core.ssl.DefaultJDKTrustConfig.createTrustManager(DefaultJDKTrustConfig.java:49) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:439) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1219) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:528) ~[?:?]
... 26 more
Caused by: java.io.IOException: DER length more than 4 bytes: 109
at org.bouncycastle.asn1.ASN1InputStream.readLength(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at org.bouncycastle.asn1.ASN1InputStream.readLength(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at org.bouncycastle.asn1.ASN1InputStream.readObject(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at org.bouncycastle.jcajce.provider.ProvBCFKS$BCFIPSKeyStoreSpi.engineLoad(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at java.security.KeyStore.load(KeyStore.java:1473) ~[?:?]
at sun.security.ssl.TrustStoreManager$TrustAnchorManager.loadKeyStore(TrustStoreManager.java:390) ~[?:?]
at sun.security.ssl.TrustStoreManager$TrustAnchorManager.getTrustedCerts(TrustStoreManager.java:336) ~[?:?]
at sun.security.ssl.TrustStoreManager.getTrustedCerts(TrustStoreManager.java:57) ~[?:?]
at sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:49) ~[?:?]
at javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:282) ~[?:?]
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.trustManager(CertParsingUtils.java:297) ~[?:?]
at org.elasticsearch.xpack.core.ssl.DefaultJDKTrustConfig.createTrustManager(DefaultJDKTrustConfig.java:49) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:439) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1219) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:528) ~[?:?]
... 26 more
[2022-04-26T14:09:38,286][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [node-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: ElasticsearchSecurityException[failed to load SSL configuration [xpack.security.transport.ssl]]; nested: ElasticsearchException[failed to initialize a TrustManagerFactory]; nested: KeyStoreException[problem accessing trust store]; nested: IOException[DER length more than 4 bytes: 109];
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:171) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:158) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:75) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:114) ~[elasticsearch-cli-7.15.1.jar:7.15.1]
at org.elasticsearch.cli.Command.main(Command.java:79) ~[elasticsearch-cli-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:123) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:81) ~[elasticsearch-7.15.1.jar:7.15.1]
Caused by: org.elasticsearch.ElasticsearchSecurityException: failed to load SSL configuration [xpack.security.transport.ssl]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:530) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1420) ~[?:?]
at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1553) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:526) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:144) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:459) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:301) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$18(Node.java:622) ~[elasticsearch-7.15.1.jar:7.15.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.<init>(Node.java:626) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.node.Node.<init>(Node.java:288) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:167) ~[elasticsearch-7.15.1.jar:7.15.1]
... 6 more
Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory
at org.elasticsearch.xpack.core.ssl.DefaultJDKTrustConfig.createTrustManager(DefaultJDKTrustConfig.java:51) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:439) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1219) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:528) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1420) ~[?:?]
at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1553) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:526) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:144) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:459) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:301) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$18(Node.java:622) ~[elasticsearch-7.15.1.jar:7.15.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.<init>(Node.java:626) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.node.Node.<init>(Node.java:288) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:167) ~[elasticsearch-7.15.1.jar:7.15.1]
... 6 more
Caused by: java.security.KeyStoreException: problem accessing trust store
at sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:73) ~[?:?]
at javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:282) ~[?:?]
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.trustManager(CertParsingUtils.java:297) ~[?:?]
at org.elasticsearch.xpack.core.ssl.DefaultJDKTrustConfig.createTrustManager(DefaultJDKTrustConfig.java:49) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:439) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1219) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:528) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1420) ~[?:?]
at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1553) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:526) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:144) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:459) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:301) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$18(Node.java:622) ~[elasticsearch-7.15.1.jar:7.15.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.<init>(Node.java:626) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.node.Node.<init>(Node.java:288) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:167) ~[elasticsearch-7.15.1.jar:7.15.1]
... 6 more
Caused by: java.io.IOException: DER length more than 4 bytes: 109
at org.bouncycastle.asn1.ASN1InputStream.readLength(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at org.bouncycastle.asn1.ASN1InputStream.readLength(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at org.bouncycastle.asn1.ASN1InputStream.readObject(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at org.bouncycastle.jcajce.provider.ProvBCFKS$BCFIPSKeyStoreSpi.engineLoad(Unknown Source) ~[bc-fips-1.0.2.jar:?]
at java.security.KeyStore.load(KeyStore.java:1473) ~[?:?]
at sun.security.ssl.TrustStoreManager$TrustAnchorManager.loadKeyStore(TrustStoreManager.java:390) ~[?:?]
at sun.security.ssl.TrustStoreManager$TrustAnchorManager.getTrustedCerts(TrustStoreManager.java:336) ~[?:?]
at sun.security.ssl.TrustStoreManager.getTrustedCerts(TrustStoreManager.java:57) ~[?:?]
at sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:49) ~[?:?]
at javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:282) ~[?:?]
at org.elasticsearch.xpack.core.ssl.CertParsingUtils.trustManager(CertParsingUtils.java:297) ~[?:?]
at org.elasticsearch.xpack.core.ssl.DefaultJDKTrustConfig.createTrustManager(DefaultJDKTrustConfig.java:49) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:439) ~[?:?]
at java.util.HashMap.computeIfAbsent(HashMap.java:1219) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.lambda$loadSSLConfigurations$5(SSLService.java:528) ~[?:?]
at java.util.HashMap.forEach(HashMap.java:1420) ~[?:?]
at java.util.Collections$UnmodifiableMap.forEach(Collections.java:1553) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:526) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.<init>(SSLService.java:144) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createSSLService(XPackPlugin.java:459) ~[?:?]
at org.elasticsearch.xpack.core.XPackPlugin.createComponents(XPackPlugin.java:301) ~[?:?]
at org.elasticsearch.node.Node.lambda$new$18(Node.java:622) ~[elasticsearch-7.15.1.jar:7.15.1]
at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:273) ~[?:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1625) ~[?:?]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
at org.elasticsearch.node.Node.<init>(Node.java:626) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.node.Node.<init>(Node.java:288) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:219) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:399) ~[elasticsearch-7.15.1.jar:7.15.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:167) ~[elasticsearch-7.15.1.jar:7.15.1]
... 6 more
It makes me think that maybe something with my conversion of the BCKFS went wrong, or i'm missing something in the java.security? Is there a known GOOD (java.security) file that should be used for FIPS and BCKFS/BCFIPS?
Also guidance here would be appreciated. We are just trying to get FIPS complaint, not really needing to certify it.