# My elasticsearch is not running with error code 128

**URL:** <https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892>\
**Category:** Elasticsearch\
**Created:** [March 17, 2023, 2:04am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892 "2023-03-17T02:04:09Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Terry\_2018](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/terry_2018/32/118545_2.png) [@Terry\_2018](https://discuss.elastic.co/u/Terry_2018)\
**Post date:** [March 17, 2023, 2:04am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/1 "2023-03-17T02:04:09Z")

</div>

Hi.  
I'm using Elasticsearch 8.6.2 on Ubuntu 22.04.  
Since a few days ago, my elastic is not running.  
Please help me.

The system output is below.

```auto
dev@logserver:~$ sudo systemctl status elasticsearch
× elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2023-03-17 09:45:13 CST; 1min 57s ago
       Docs: https://www.elastic.co
    Process: 2133 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=128)
   Main PID: 2133 (code=exited, status=128)
        CPU: 1min 30.382s

Mar 17 09:44:43 logserver systemd[1]: Starting Elasticsearch...
Mar 17 09:45:03 logserver systemd[1]: Started Elasticsearch.
Mar 17 09:45:13 logserver systemd-entrypoint[2133]: ERROR: Elasticsearch exited unexpectedly
Mar 17 09:45:13 logserver systemd[1]: elasticsearch.service: Main process exited, code=exited, status=128/n/a
Mar 17 09:45:13 logserver systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 17 09:45:13 logserver systemd[1]: elasticsearch.service: Consumed 1min 30.382s CPU time.

```

And I have elasticsearch.log, but I don't know how can share it.

Please help me.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 17, 2023, 2:46am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/2 "2023-03-17T02:46:48Z")

</div>

Hi @Terry_2018 Welcome to the community!

Pastebin... Or a gist

Try running

`journalctl -u elasticsearch -f`

---

<div class="post-metadata">

**Author:** ![Terry\_2018](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/terry_2018/32/118545_2.png) [@Terry\_2018](https://discuss.elastic.co/u/Terry_2018)\
**Post date:** [March 17, 2023, 3:13am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/3 "2023-03-17T03:13:24Z")

</div>

```auto
Mar 17 09:41:59 logserver systemd-entrypoint[1876]: ERROR: Elasticsearch exited unexpectedly
Mar 17 09:41:59 logserver systemd[1]: elasticsearch.service: Main process exited, code=exited, status=128/n/a
Mar 17 09:41:59 logserver systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 17 09:41:59 logserver systemd[1]: elasticsearch.service: Consumed 1min 36.959s CPU time.
Mar 17 09:44:43 logserver systemd[1]: Starting Elasticsearch...
Mar 17 09:45:03 logserver systemd[1]: Started Elasticsearch.
Mar 17 09:45:13 logserver systemd-entrypoint[2133]: ERROR: Elasticsearch exited unexpectedly
Mar 17 09:45:13 logserver systemd[1]: elasticsearch.service: Main process exited, code=exited, status=128/n/a
Mar 17 09:45:13 logserver systemd[1]: elasticsearch.service: Failed with result 'exit-code'.
Mar 17 09:45:13 logserver systemd[1]: elasticsearch.service: Consumed 1min 30.382s CPU time.

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [March 17, 2023, 3:21am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/4 "2023-03-17T03:21:45Z")

</div>

You won't see the error with journalctl, you need to look directly in the syslog file, check `/var/log/syslog` for any hint of why the systemd server cannot start the elasticsearch service.

You may try to start it again to get some fresh logs.

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 17, 2023, 3:33am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/5 "2023-03-17T03:33:55Z")

</div>

> [@leandrojmp](#):
>
> You won't see the error with journalctl

Yup you are right ... other components do elasticsearch not...

Assuming you installed with defaults...

`tail -f /var/log/elasticsearch/elasticsearch.log`

---

<div class="post-metadata">

**Author:** ![Terry\_2018](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/terry_2018/32/118545_2.png) [@Terry\_2018](https://discuss.elastic.co/u/Terry_2018)\
**Post date:** [March 17, 2023, 4:27am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/6 "2023-03-17T04:27:09Z")

</div>

I have entire elasticsearch.log, how can I share it?

---

<div class="post-metadata">

**Author:** ![Terry\_2018](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/terry_2018/32/118545_2.png) [@Terry\_2018](https://discuss.elastic.co/u/Terry_2018)\
**Post date:** [March 17, 2023, 7:08am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/7 "2023-03-17T07:08:10Z")

</div>

> [@stephenb](#):
>
> tail -f /var/log/elasticsearch/elasticsearch.log

```auto
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
		at java.lang.Thread.run(Thread.java:1589) ~[?:?]
	Caused by: java.lang.InternalError: a fault occurred in an unsafe memory access operation
		at org.apache.lucene.store.BufferedChecksumIndexInput.readBytes(BufferedChecksumIndexInput.java:46) ~[lucene-core-9.4.2.jar:?]
		at org.apache.lucene.store.DataInput.readBytes(DataInput.java:72) ~[lucene-core-9.4.2.jar:?]
		at org.apache.lucene.store.ChecksumIndexInput.skipByReading(ChecksumIndexInput.java:79) ~[lucene-core-9.4.2.jar:?]
		at org.apache.lucene.store.ChecksumIndexInput.seek(ChecksumIndexInput.java:64) ~[lucene-core-9.4.2.jar:?]
		at org.apache.lucene.store.IndexInput.skipBytes(IndexInput.java:89) ~[lucene-core-9.4.2.jar:?]
		at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:489) ~[lucene-core-9.4.2.jar:?]
		... 22 more

```

---

<div class="post-metadata">

**Author:** ![Terry\_2018](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/terry_2018/32/118545_2.png) [@Terry\_2018](https://discuss.elastic.co/u/Terry_2018)\
**Post date:** [March 17, 2023, 10:20am UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/8 "2023-03-17T10:20:28Z")

</div>

> <https://gist.github.com/terry1102/afc8c676087ecbe2da9d87c461dd2d1f>

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [March 17, 2023, 3:03pm UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/9 "2023-03-17T15:03:52Z")

</div>

Hi @Terry_2018

I think we need to back up.

Exactly how did you install?

How many nodes?

Is security Enabled?

What is the size of the server? CPU RAM

Are you running other applications / processes?

Did you do anything in the data path? Move files touch them etc?

It looks like there are corrupted indices / data

---

<div class="post-metadata">

**Author:** ![William\_Brafford](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/william_brafford/32/51559_2.png) [@William\_Brafford](https://discuss.elastic.co/u/William_Brafford)\
**Post date:** [March 20, 2023, 6:54pm UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/10 "2023-03-20T18:54:02Z")

</div>

That log has several errors with a cause like this:

```auto
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[.geoip_databases][0]]. Consider using `allow_partial_search_results` setting to bypass this error.
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:245) ~[elasticsearch-8.6.2.jar:?]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:469) ~[elasticsearch-8.6.2.jar:?]
	... 190 more

```

or

```auto
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: Search rejected due to missing shards [[metrics-endpoint.metadata_current_default][0]]. Consider using `allow_partial_search_results` setting to bypass this error.
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:245) ~[elasticsearch-8.6.2.jar:?]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:469) ~[elasticsearch-8.6.2.jar:?]
	... 97 more

```

I also suspect that files in the node's data path may have been modified or deleted.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 17, 2023, 6:54pm UTC](https://discuss.elastic.co/t/my-elasticsearch-is-not-running-with-error-code-128/327892/11 "2023-04-17T18:54:23Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
