Elasticsearch.service: Main process exited, code=exited, status=1/FAILURE when setting TLS

[derekmizak (DerekM)]

ES 7.2
When following article about setting up cluster security https://www.elastic.co/blog/getting-started-with-elasticsearch-security

I am unable to start elasticsearch instance, after adding the following to elasticsearch.yml
xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate

xpack.security.transport.ssl.keystore.path: clustercerts.p12

xpack.security.transport.ssl.truststore.path: clustercerts.p12

There is nothing in the log - the only error is when checking status of the service:
systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE

I would appreciate any help with diagnostics.

[spinscale (Alexander Reelsen)]

hey,

can you provide the whole error message, either in the systemd logs or in the elasticsearch logs? Right now I am assuming, that the path to the p12 file is wrong, as it needs to refer to the path within the config directory? Maybe those certs have been put in a sub directory? Or maybe something in those cert files is missing - thus we need more information regarding the error message.

--Alex

[derekmizak (DerekM)]

Hi,

Certificate file is located in /usr/share/elasticsearch/clustercerts.p.12

elasticsearch.yml has the following added:

xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true

xpack.security.transport.ssl.verification_mode: certificate

xpack.security.transport.ssl.keystore.path: clustercerts.p12

xpack.security.transport.ssl.truststore.path: clustercerts.p12

Passwords for the certificates have been added using the following command:
elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password

Certificates have been created as follow:
First CA has been created:
sudo /usr/share/elasticsearch/bin/elasticsearch-certutil ca

After that certificates:
sudo /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca firm-rblog-ca-cert.p12 --out clustercerts.p12 --pass *************

The only errors are visible using systemctl status elasticsearch.service

Process: 12515 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)

 Main PID: 12515 (code=exited, status=1/FAILURE)

Jun 27 16:42:44 firm-rbl01 systemd[1]: Started Elasticsearch.

Jun 27 16:42:45 firm-rbl01 elasticsearch[12515]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be remo

Jun 27 16:42:48 firm-rbl01 systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE

Jun 27 16:42:48 firm-rbl01 systemd[1]: elasticsearch.service: Failed with result 'exit-code'.

I cant locate any other errors.

[derekmizak (DerekM)]

HI,
Got a bit more info.

My permissions on cert files are as follow:
-rwxrwxr-x 1 root elasticsearch 3443 Jun 27 17:39 clustercerts.p12
-rwxrwxr-x 1 root root 2527 Jun 27 16:11 firm-rblog-ca-cert.p12

However, I was able to find some permission issues in the cluster log

**Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/usr/share/elasticsearch/clustercerts.p12" "read")**

More of the listing below:

[2019-06-27T18:11:26,107][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [firm-rbl01] uncaught exception in thread [main]

org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin]

....
Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory
at org.elasticsearch.xpack.core.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:61) ~[?:?]
at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:382) ~[?:?]

....

Caused by: java.security.AccessControlException: access denied ("java.io.FilePermission" "/usr/share/elasticsearch/clustercerts.p12" "read")
	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
	at java.security.AccessController.checkPermission(AccessController.java:1042) ~[?:?]
	at java.lang.SecurityManager.checkPermission(SecurityManager.java:408) ~[?:?]
	at java.lang.SecurityManager.checkRead(SecurityManager.java:747) ~[?:?]
	at sun.nio.fs.UnixChannelFactory.open(UnixChannelFactory.java:255) ~[?:?]

Why is there permission issue when everyone has read access at the moment.

[derekmizak (DerekM)]

I may have found a solution to the error issue.

I am working on Ubuntu and cluster is installed from the .deb packages. Certificates have to be placed in /etc/elasticsearch/ directory and not in the /usr/share/elasticsearch.

Once moved it works well.