Elasticsearch version 6.8.6 oss unable to start

Elastic Stack Elasticsearch
1

Hi all,

I am trying to do a fresh install of elasticsearch-oss-6.8.6 with OpenJDK version 1.8.0_181. An elasticsearch-oss-6.5 was able to successfully install before but I am receiving errors related to the keystore and trust manager. There were no issues related to the trust manager and keystore before.

Here is the exception from elasticsearch.log:

[2020-01-13T19:11:26,760][INFO ][o.e.p.PluginsService     ] [xSBU6-2] no plugins loaded
[2020-01-13T19:11:29,780][ERROR][o.e.b.Bootstrap          ] [xSBU6-2] Exception
org.elasticsearch.common.ssl.SslConfigException: failed to initialize a TrustManager for the system keystore
	at org.elasticsearch.common.ssl.DefaultJdkTrustConfig.createTrustManager(DefaultJdkTrustConfig.java:70) ~[?:?]
	at org.elasticsearch.common.ssl.SslConfiguration.createSslContext(SslConfiguration.java:109) ~[?:?]
	at org.elasticsearch.index.reindex.ReindexSslConfig.reload(ReindexSslConfig.java:145) ~[?:?]
	at org.elasticsearch.index.reindex.ReindexSslConfig.<init>(ReindexSslConfig.java:115) ~[?:?]
	at org.elasticsearch.index.reindex.ReindexPlugin.createComponents(ReindexPlugin.java:88) ~[?:?]
	at org.elasticsearch.node.Node.lambda$new$11(Node.java:472) ~[elasticsearch-6.8.6.jar:6.8.6]
	at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:267) ~[?:1.8.0_181]
	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_181]
	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_181]
	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_181]
	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_181]
	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_181]
	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_181]
	at org.elasticsearch.node.Node.<init>(Node.java:475) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.6.jar:6.8.6]
Caused by: java.security.KeyStoreException: problem accessing trust storejava.io.IOException: Invalid keystore format
	at sun.security.ssl.TrustManagerFactoryImpl.engineInit(TrustManagerFactoryImpl.java:74) ~[?:?]
	at javax.net.ssl.TrustManagerFactory.init(TrustManagerFactory.java:250) ~[?:1.8.0_181]
	at org.elasticsearch.common.ssl.KeyStoreUtil.createTrustManager(KeyStoreUtil.java:151) ~[?:?]
	at org.elasticsearch.common.ssl.DefaultJdkTrustConfig.createTrustManager(DefaultJdkTrustConfig.java:68) ~[?:?]
	... 24 more

Not sure if this will help but this is from "systemctl status elasticsearch":

[red dot] elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2020-01-14 08:34:19 UTC; 8s ago
Docs: http://www.elastic.co
Process: 5553 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
Main PID: 5553 (code=exited, status=1/FAILURE)

Jan 14 08:34:13 thetemplate systemd[1]: Started Elasticsearch.
Jan 14 08:34:13 thetemplate systemd[1]: Starting Elasticsearch...
Jan 14 08:34:19 thetemplate systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jan 14 08:34:19 thetemplate systemd[1]: Unit elasticsearch.service entered failed state.
Jan 14 08:34:19 thetemplate systemd[1]: elasticsearch.service failed.

2

The OSS distribution of Elasticsearch does not include support for security so I am not sure this is expected to work. With the default distribution security is now available as part of the basic license though.

Did you have any security plugin installed on Elasticsearch-oss 6.5?

3

Hi Christian I'll be driving this for Brandon today.

Can you elaborate on why this is not expected to work? We think this should work just fine.
We are using the OSS 6.8.6 RPM downloaded from here:
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.8.6.rpm

Previously, we were using the OSS 6.5.4 RPM downloaded from here:
https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-oss-6.5.4.rpm

We are not trying to perform any kind of upgrade on Elasticsearch. We are taking our base SW product and installing the OSS 6.8.6 RPM on top of it which worked just fine with OSS 6.5.4
We have never used Xpack or any other security plugin for Elasticsearch.
We see this keystore exception thrown everytime Elasticsearch attempts to start.

The only things I have found online relating to this are:

Which seem to suggest it is an OracleJDK vs OpenJDK problem. As Brandon described we are using OpenJDK and have no way currently of using OracleJDK, but the support matrix states that OpenJDK is supported for this version of Elasticsearch.

4

I mentioned it at the top of the post but at the moment, we are currently using OpenJDK version 1.8.0_181 and it seems like this is the supported version in https://www.elastic.co/support/matrix#matrix_jvm. Is this link outdated or is this link specifically for non-oss versions?

5

This issue never came up with version 6.5.4 but I believe this was mainly because there wasn't a /libs/ssl-config directory and for some reason why, the OSS version is calling this library when there isn't any security or xpack configurations.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.