# Snapshot failed with "file\_system\_exception... Structure needs cleaning"

**URL:** <https://discuss.elastic.co/t/snapshot-failed-with-file-system-exception-structure-needs-cleaning/231200>\
**Category:** Elasticsearch\
**Tags:** slm-snapshot-lifecycle-management\
**Created:** [May 5, 2020, 4:57pm UTC](https://discuss.elastic.co/t/snapshot-failed-with-file-system-exception-structure-needs-cleaning/231200 "2020-05-05T16:57:24Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Davoot](https://avatars.discourse-cdn.com/v4/letter/d/f1d935/32.png) [@Davoot](https://discuss.elastic.co/u/Davoot)\
**Post date:** [May 5, 2020, 4:57pm UTC](https://discuss.elastic.co/t/snapshot-failed-with-file-system-exception-structure-needs-cleaning/231200/1 "2020-05-05T16:57:25Z")

</div>

Hi all,  
I'm trying to snapshot an index to an fs repository (nfs share).  
I have two nodes in 7.6.1. First node share nfs, second mount it  
There is around 80 indices (30 GB/day), I decide to snapshot monthly manually before setting an ILM that will delete cold index after 30 days. I need to keep backup for 1 year  
First month run smoothly and snapshot 18 index, other dosen't and report error bellow in police history

```
   {
  "type": "snapshot_exception",
  "reason": "[faz_archives:snapshot-faz-fj2wco52rh2b-g0dgjq1hw/CCvUmPNxTfSQe9mQKyphbQ] failed to update snapshot in repository",
  "caused_by": {
"type": "file_system_exception",
"reason": "/var/archives/pending-index.latest-g7Kq4PgbTtq2-Uz1JW3B6A: Structure needs cleaning",
"stack_trace": "java.nio.file.FileSystemException: /var/archives/pending-index.latest-g7Kq4PgbTtq2-Uz1JW3B6A: Structure needs cleaning\n\tat sun.nio.fs.UnixException.translateToIOException(UnixException.java:91)\n\tat sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)\n\tat sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)\n\tat sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)\n\tat java.nio.file.spi.FileSystemProvider.newOutputStream(FileSystemProvider.java:434)\n\tat java.nio.file.Files.newOutputStream(Files.java:216)\n\tat org.elasticsearch.common.blobstore.fs.FsBlobContainer.writeBlobAtomic(FsBlobContainer.java:174)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.writeAtomic(BlobStoreRepository.java:1455)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.lambda$writeIndexGen$47(BlobStoreRepository.java:1335)\n\tat org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63)\n\tat org.elasticsearch.common.util.concurrent.ListenableFuture$1.doRun(ListenableFuture.java:112)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\n\tat org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService.execute(EsExecutors.java:225)\n\tat org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListener(ListenableFuture.java:106)\n\tat org.elasticsearch.common.util.concurrent.ListenableFuture.lambda$done$0(ListenableFuture.java:98)\n\tat java.util.ArrayList.forEach(ArrayList.java:1257)\n\tat org.elasticsearch.common.util.concurrent.ListenableFuture.done(ListenableFuture.java:98)\n\tat org.elasticsearch.common.util.concurrent.BaseFuture.set(BaseFuture.java:144)\n\tat org.elasticsearch.common.util.concurrent.ListenableFuture.onResponse(ListenableFuture.java:127)\n\tat org.elasticsearch.action.StepListener.innerOnResponse(StepListener.java:62)\n\tat org.elasticsearch.action.NotifyOnceListener.onResponse(NotifyOnceListener.java:40)\n\tat org.elasticsearch.repositories.blobstore.BlobStoreRepository.lambda$writeIndexGen$45(BlobStoreRepository.java:1312)\n\tat org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:692)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:748)\n"
},
...
}

```

There are folders created in repository, but I don't see my snapshots.  
Also, Data drive is growing each time I launch snapshot policy !  
How can I clean structure as requested and recover data drive space ?  
Thanks for help

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [May 5, 2020, 6:17pm UTC](https://discuss.elastic.co/t/snapshot-failed-with-file-system-exception-structure-needs-cleaning/231200/2 "2020-05-05T18:17:06Z")

</div>

> [@Davoot](#):
>
> `Structure needs cleaning`

This error is coming from your operating system and [suggests that your filesystem may be corrupt](https://unix.stackexchange.com/questions/330742/cannot-remove-file-structure-needs-cleaning). I don't think there's much that Elasticsearch can do here, this problem lies elsewhere.

---

<div class="post-metadata">

**Author:** ![Davoot](https://avatars.discourse-cdn.com/v4/letter/d/f1d935/32.png) [@Davoot](https://discuss.elastic.co/u/Davoot)\
**Post date:** [May 5, 2020, 8:04pm UTC](https://discuss.elastic.co/t/snapshot-failed-with-file-system-exception-structure-needs-cleaning/231200/3 "2020-05-05T20:04:59Z")

</div>

You're right,  
After an xfs\_repair, snapshot are running again.. Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 2, 2020, 8:05pm UTC](https://discuss.elastic.co/t/snapshot-failed-with-file-system-exception-structure-needs-cleaning/231200/4 "2020-06-02T20:05:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.