ES not starting : After restarting elasticsearch 7.6.2 throws CorruptIndexException

Elastic Stack Elasticsearch
1

I am unable to restart my elasticsearch 7.6.2 as I am getting below exception, Can someone help me out regarding this issue.

ELK elasticsearch[3805]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely May 08 01:53:49 ELK elasticsearch[3805]: Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: org.apache.lucene.index.Corrup May 08 01:53:49 ELK elasticsearch[3805]: Likely root cause: org.apache.lucene.index.CorruptIndexException: misplaced codec footer (file extended?) May 08 01:53:49 ELK elasticsearch[3805]: at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:497) May 08 01:53:49 ELK elasticsearch[3805]: at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:414) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:282) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:234) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:305) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:125) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.cli.Command.main(Command.java:90) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) May 08 01:53:49 ELK elasticsearch[3805]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) May 08 01:53:49 ELK systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE May 08 01:53:49 ELK systemd[1]: elasticsearch.service: Failed with result 'exit-code'. May 08 01:53:49 ELK systemd[1]: Failed to start Elasticsearch. -- Subject: Unit elasticsearch.service has failed

2

Hi @akELK081 - Are you able to provide the relevant (and complete) Elasticsearch logs? I believe the above logs were truncated (not sure if it is a problem with the copy/paste).

3

Hi @ropc actually I tried to enable xpack security in elasticsearch.yml, without stopping the kibana service, After modifying elasticsearch, I restarted elasticsearch service but it is always failed to start throwing CorruptIndexException followed with above attached logs.

4

This means that something outside of Elasticsearch has changed this file since Elasticsearch wrote it. Normally it's either a hardware problem or some other rogue piece of software meddling with things it's not supposed to. Either way, that's very bad. Once you've worked out what caused that and stopped it from happening again the best fix is to wipe this node and start it afresh with an empty data path, allowing Elasticsearch to recover any lost shards from other nodes in the cluster and/or recovering them from a recent snapshot.

5

In addition to what @DavidTurner has mentioned, you could also try to use the elasticsearch-shard tool to remove the corrupted data.

6

Thank you @DavidTurner and @ropc for provide possible approach to tackle these scenario. I just wiped it and started it fresh.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.