# My ela cluster failed to load metadata when running, but I can't see the other problem. If there is the same problem, please help to take a look **URL:** \ **Category:** Elasticsearch\ **Created:** [February 23, 2023, 8:47am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286 "2023-02-23T08:47:00Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![limedong1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limedong1/32/117628_2.png) [@limedong1](https://discuss.elastic.co/u/limedong1)\ **Post date:** [February 23, 2023, 8:47am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/1 "2023-02-23T08:47:01Z")
- Here is the error message： {"@timestamp":"2023-02-23T08:43:40.767Z", "log.level":"ERROR", "message":"fatal exception while booting Elasticsearch", "ecs.version": "1.2.0","service.name":"ES\_ECS","event.dataset":"elasticsearch.server","process.thread.name":"main","log.logger":"org.elasticsearch.bootstrap.Elasticsearch","elasticsearch.node.name":"elastic-cluster-0","elasticsearch.cluster.name":"elasticsearch-cluster","error.type":"org.elasticsearch.ElasticsearchException","error.message":"failed to load metadata","error.stack\_trace":"org.elasticsearch.ElasticsearchException: failed to load metadata\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.gateway.GatewayMetaState.start(GatewayMetaState.java:161)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.node.Node.start(Node.java:1354)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.bootstrap.Elasticsearch.start(Elasticsearch.java:436)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:229)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:67)\nCaused by: org.apache.lucene.index.CorruptIndexException: Unexpected file read error while reading index. (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/usr/share/elasticsearch/data/\_state/segments\_3")))\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:301)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:288)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.IndexFileDeleter.(IndexFileDeleter.java:166)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.IndexWriter.(IndexWriter.java:1158)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.gateway.PersistedClusterStateService.createIndexWriter(PersistedClusterStateService.java:264)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.gateway.PersistedClusterStateService.createWriter(PersistedClust erStateService.java:226)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.gateway.GatewayMetaState$LucenePersistedState.(GatewayMetaState.java:447)\n\tat org.elasticsearch.server@8.6.0/org.elasticsearch.gateway.GatewayMetaState.start(GatewayMetaState.java:130)\n\t... 4 more\nCaused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/\_state/\_1.si\n\tat java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)\n\tat java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)\n\tat java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:181)\n\tat java.base/java.nio.channels.FileChannel.open(FileChannel.java:304)\n\tat java.base/java.nio.channels.FileChannel.open(FileChannel.java:363)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.codecs.lucene90.Lucene90SegmentInfoFormat.read(Lucene90SegmentInfoFormat.java:102)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.SegmentInfos.parseSegmentInfos(SegmentInfos.java:406)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:363)\n\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:299)\n\t... 11 more\n\tSuppressed: org.apache.lucene.index.CorruptIndexException: checksum passed (39cc8ebc). possibly transient resource issue, or a Lucene or JVM bug (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="/usr/share/elasticsearch/data/\_state/segments\_3")))\n\t\tat org.apache.lucene.core@9.4.2/org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:500)\n\t\tat org.apache.lucene.core@9.4.2/org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:370)\n\t\ t... 12 more\n"} There are three machines in the cluster, each of which is on, and I am confident that using the single node before the cluster is OK ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 23, 2023, 8:52am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/2 "2023-02-23T08:52:48Z")
Please don't only post an error message with no other information. [Dec 10th, 2022: [EN] Asking top notch technical questions to get you help quicker!](https://discuss.elastic.co/t/dec-10th-2022-en-asking-top-notch-technical-questions-to-get-you-help-quicker/320300) has some guidance on what to provide. ---
**Author:** ![limedong1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limedong1/32/117628_2.png) [@limedong1](https://discuss.elastic.co/u/limedong1)\ **Post date:** [February 23, 2023, 9:13am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/3 "2023-02-23T09:13:38Z")
oh sorry，my mistke ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 23, 2023, 9:22pm UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/4 "2023-02-23T21:22:37Z")
What version of Elasticsearch are you using? ---
**Author:** ![limedong1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limedong1/32/117628_2.png) [@limedong1](https://discuss.elastic.co/u/limedong1)\ **Post date:** [February 24, 2023, 1:13am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/5 "2023-02-24T01:13:48Z")
elasticsearch image version is 8.6.0 ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [February 24, 2023, 10:10am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/6 "2023-02-24T10:10:13Z")
A required file is missing: ```plaintext Caused by: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/data/_state/_1.si ``` Elasticsearch didn't delete this file, so you have some problem with your storage or something else removed this file. If the rest of the cluster is healthy then the simplest fix is to wipe this node and reinstall from scratch, but you must work out what caused this file to go missing or else it will just happen again. ---
**Author:** ![limedong1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limedong1/32/117628_2.png) [@limedong1](https://discuss.elastic.co/u/limedong1)\ **Post date:** [February 27, 2023, 1:35am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/8 "2023-02-27T01:35:32Z")
> [@DavidTurner](#): > > Elasticsearch didn't delete this file, so you have some problem with your storage or something else removed this file. If the rest of the cluster is healthy then the simplest fix is to wipe this node and reinstall from scratch, but you must work out what caused this file to go missing or else it will just happen again. Thanks for your reply, David, because I am only installing elasticserach now, the entire cluster of nodes are reporting this error, and the other nodes are also unhealthy. My storage is the remote storage OSS provided by the preparator. I think it should be that the cluster node file is overwritten, which means that maybe only one node's file is written, and the other two nodes' files are directly overwritten, resulting in constant reporting of missing files. ---
**Author:** ![limedong1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limedong1/32/117628_2.png) [@limedong1](https://discuss.elastic.co/u/limedong1)\ **Post date:** [February 27, 2023, 3:14am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/9 "2023-02-27T03:14:38Z")
The problem was solved. The pain point of the problem was that I did not fix path.data for each node, resulting in all the files of the node being written to the same directory. ---
**Author:** ![limedong1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/limedong1/32/117628_2.png) [@limedong1](https://discuss.elastic.co/u/limedong1)\ **Post date:** [February 27, 2023, 3:17am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/10 "2023-02-27T03:17:01Z")
By the way, I did not set up my elasticsearch cluster locally; I set it up using k8s ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 27, 2023, 3:17am UTC](https://discuss.elastic.co/t/my-ela-cluster-failed-to-load-metadata-when-running-but-i-cant-see-the-other-problem-if-there-is-the-same-problem-please-help-to-take-a-look/326286/11 "2023-03-27T03:17:28Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.