# Elasticsearch startup problemse

**URL:** <https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831>\
**Category:** Elasticsearch\
**Tags:** elastic-stack-security\
**Created:** [March 4, 2022, 9:49am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831 "2022-03-04T09:49:49Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![zx\_emo](https://avatars.discourse-cdn.com/v4/letter/z/e19adc/32.png) [@zx\_emo](https://discuss.elastic.co/u/zx_emo)\
**Post date:** [March 4, 2022, 9:49am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831/1 "2022-03-04T09:49:49Z")

</div>

Hello, my Yml file looks like this

```auto
#xpack.license.self_generated.type: basic
xpack.security.enabled: true

#xpack.security.enrollment.enabled: true
#xpack.security.enrollment.enabled: true

# Enable encryption for HTTP API client connections, such as Kibana, Logstash, and Agents
#xpack.security.http.ssl.enabled: true
#keystore.path: /usr/software/es/elasticsearch-8.0.1/config/certs/http.p12
xpack.security.transport.ssl.enabled: true
#
xpack.security.transport.ssl:
  enabled: true
  verification_mode: certificate
  client_authentication: required
  keystore.path: /usr/software/es/elasticsearch-8.0.1/config/certs/elastic-certificates.p12
  truststore.path: /usr/software/es/elasticsearch-8.0.1/config/certs/elastic-certificates.p12
#
#xpack.security.transport.ssl.client_authentication: required
#
#xpack.security.transport.ssl.keystore.path: /usr/software/es/elasticsearch-8.0.1/config/certs/elastic-certificates.p12
#
#xpack.security.transport.ssl.truststore.path: /usr/software/es/elasticsearch-8.0.1/config/certs/elastic-certificates.p12
#
#cluster.initial_master_nodes: ["localhost.localdomain"]

discovery.type: single-node

```

When I started Elasticsearch with the above configuration I got an error with the following message

```auto
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]
        at sun.security.ssl.TransportContext.fatal(TransportContext.java:371) ~[?:?]
        at sun.security.ssl.TransportContext.fatal(TransportContext.java:314) ~[?:?]
        at sun.security.ssl.TransportContext.fatal(TransportContext.java:309) ~[?:?]
        at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654) ~[?:?]
        at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473) ~[?:?]
        at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369) ~[?:?]
        at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]
        at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:480) ~[?:?]
        at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:458) ~[?:?]
        at sun.security.ssl.TransportContext.dispatch(TransportContext.java:201) ~[?:?]
        at sun.security.ssl.SSLTransport.decode(SSLTransport.java:172) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450) ~[?:?]
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421) ~[?:?]
        at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580) ~[?:?]
        at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) ~[?:?]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665) ~[?:?]
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) ~[?:?]
        at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) ~[?:?]
        at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) ~[?:?]
        at org.elasticsearch.ingest.geoip.HttpClient.lambda$get$0(HttpClient.java:46) ~[ingest-geoip-8.0.1.jar:8.0.1]
        at java.security.AccessController.doPrivileged(AccessController.java:569) ~[?:?]
        at org.elasticsearch.ingest.geoip.HttpClient.doPrivileged(HttpClient.java:88) ~[ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.ingest.geoip.HttpClient.get(HttpClient.java:40) ~[ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.ingest.geoip.HttpClient.getBytes(HttpClient.java:36) ~[ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.ingest.geoip.GeoIpDownloader.fetchDatabasesOverview(GeoIpDownloader.java:143) ~[ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:131) ~[ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:265) [ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:100) [ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:46) [ingest-geoip-8.0.1.jar:8.0.1]
        at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:42) [elasticsearch-8.0.1.jar:8.0.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:776) [elasticsearch-8.0.1.jar:8.0.1]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-8.0.1.jar:8.0.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.lang.Thread.run(Thread.java:833) [?:?]
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) ~[?:?]
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) ~[?:?]
        at sun.security.validator.Validator.validate(Validator.java:264) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:?]
        at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638) ~[?:?]
        ... 33 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
        at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) ~[?:?]
        at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) ~[?:?]
        at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297) ~[?:?]
        at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434) ~[?:?]
        at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) ~[?:?]
        at sun.security.validator.Validator.validate(Validator.java:264) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231) ~[?:?]
        at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:132) ~[?:?]
        at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:638) ~[?:?]
        ... 33 more

```

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [March 7, 2022, 12:09am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831/2 "2022-03-07T00:09:17Z")

</div>

In the [log files](https://www.elastic.co/guide/en/elasticsearch/reference/8.0/logging.html), just above the error traces, there should be a `WARN` message for `DiagnosticTrustManager`.  
Can you paste that message here please?

---

<div class="post-metadata">

**Author:** ![zx\_emo](https://avatars.discourse-cdn.com/v4/letter/z/e19adc/32.png) [@zx\_emo](https://discuss.elastic.co/u/zx_emo)\
**Post date:** [March 7, 2022, 5:32am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831/3 "2022-03-07T05:32:15Z")

</div>

@TimV

I'm not sure what I found was what you wanted

```auto
[WARN][o.e.c.r.a. DiskThresholdMonitor] [localhost. localdomain] flood stage disk watermark [95%] exceeded on [-DzqsBQRTuG-bo8gVHgkDA][localhost.localdomain][/usr/software/es/elasticsearch-8.0.1/data] free: 1.3gb[2.7%], all indices on this node will be marked read-only 
[ERROR][o.e.i.g.GeoIpDownloader] [localhost.localdomain] exception during geoip databases update
java.net.SocketTimeoutException: Read timed out
	at sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:283) ~[?:?]
	at sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:309) ~[?:?]
	at sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:350) ~[?:?]
	at sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:803) ~[?:?]
	at java.net.Socket$SocketInputStream.read(Socket.java:966) ~[?:?]
	at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478) ~[?:?]
	at sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472) ~[?:?]
	at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:160) ~[?:?]
	at sun.security.ssl.SSLTransport.decode(SSLTransport.java:111) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1500) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1415) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450) ~[?:?]
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:421) ~[?:?]
	at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:580) ~[?:?]
	at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1665) ~[?:?]
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1589) ~[?:?]
	at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:529) ~[?:?]
	at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:308) ~[?:?]
	at org.elasticsearch.ingest.geoip.HttpClient.lambda$get$0(HttpClient.java:46) ~[ingest-geoip-8.0.1.jar:8.0.1]
	at java.security.AccessController.doPrivileged(AccessController.java:569) ~[?:?]
	at org.elasticsearch.ingest.geoip.HttpClient.doPrivileged(HttpClient.java:88) ~[ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.ingest.geoip.HttpClient.get(HttpClient.java:40) ~[ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.ingest.geoip.HttpClient.getBytes(HttpClient.java:36) ~[ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.fetchDatabasesOverview(GeoIpDownloader.java:143) ~[ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.updateDatabases(GeoIpDownloader.java:131) ~[ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloader.runDownloader(GeoIpDownloader.java:265) [ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:100) [ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.ingest.geoip.GeoIpDownloaderTaskExecutor.nodeOperation(GeoIpDownloaderTaskExecutor.java:46) [ingest-geoip-8.0.1.jar:8.0.1]
	at org.elasticsearch.persistent.NodePersistentTasksExecutor$1.doRun(NodePersistentTasksExecutor.java:42) [elasticsearch-8.0.1.jar:8.0.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:776) [elasticsearch-8.0.1.jar:8.0.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-8.0.1.jar:8.0.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
	at java.lang.Thread.run(Thread.java:833) [?:?]
[2022-03-07T00:24:57,620][WARN][o.e.c.r.a.DiskThresholdMonitor] [localhost.localdomain] flood stage disk watermark [95%] exceeded on [-DzqsBQRTuG-bo8gVHgkDA][localhost.localdomain][/usr/software/es/elasticsearch-8.0.1/data] free: 1.3gb[2.7%], all indices on this node will be marked read-only
[2022-03-07T00:25:27,630][WARN][o.e.c.r.a.DiskThresholdMonitor] [localhost.localdomain] flood stage disk watermark [95%] exceeded on [-DzqsBQRTuG-bo8gVHgkDA][localhost.localdomain][/usr/software/es/elasticsearch-8.0.1/data] free: 1.3gb[2.7%], all indices on this node will be marked read-only
[2022-03-07T00:25:32,250][INFO][o.e.x.m.p.NativeController] [localhost.localdomain] Native controller process has stopped - no new native processes can be started

```

---

<div class="post-metadata">

**Author:** ![amitmbm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/amitmbm/32/102111_2.png) [@amitmbm](https://discuss.elastic.co/u/amitmbm)\
**Post date:** [March 7, 2022, 5:59am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831/4 "2022-03-07T05:59:38Z")

</div>

You don't have much disk space left on node running the Elasticsearch process, as mentioned in the below log.

`[WARN][o.e.c.r.a. DiskThresholdMonitor] [localhost. localdomain] flood stage disk watermark [95%] exceeded on [-DzqsBQRTuG-bo8gVHgkDA][localhost.localdomain][/usr/software/es/elasticsearch-8.0.1/data] free: 1.3gb[2.7%], all indices on this node will be marked read-only`

Could you delete unused files to free up space, which is important as disk watermark reached flood state, it means you won't be able to write anything new in your Elasticsearch.

Also, your earlier errors were related to security configs, So were you able to fix it, if you want you can just enable the basic security to make sure you are able to up the service.

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [March 7, 2022, 6:38am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831/5 "2022-03-07T06:38:17Z")

</div>

Please check the logs for the words `DiagnosticTrustManager`  
There should be a message a few milliseconds before the `SSLHandshakeException` you posted.

---

<div class="post-metadata">

**Author:** ![zx\_emo](https://avatars.discourse-cdn.com/v4/letter/z/e19adc/32.png) [@zx\_emo](https://discuss.elastic.co/u/zx_emo)\
**Post date:** [March 7, 2022, 7:20am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831/6 "2022-03-07T07:20:59Z")

</div>

> [@TimV](#):
>
> DiagnosticTrustManager

I did not find the prompt you said whether it does not exist with elasticSearch.log

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/5/45209defd7353fe42956b77ce6a17731dc87b9af.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 4, 2022, 7:21am UTC](https://discuss.elastic.co/t/elasticsearch-startup-problemse/298831/7 "2022-04-04T07:21:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.