# Snapshots on NFS4 intermittently reporting PARTIAL

**URL:** <https://discuss.elastic.co/t/snapshots-on-nfs4-intermittently-reporting-partial/265800>\
**Category:** Elasticsearch\
**Tags:** snapshot-and-restore\
**Created:** [March 1, 2021, 1:19pm UTC](https://discuss.elastic.co/t/snapshots-on-nfs4-intermittently-reporting-partial/265800 "2021-03-01T13:19:47Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![ArturGrzella](https://avatars.discourse-cdn.com/v4/letter/a/b5a626/32.png) [@ArturGrzella](https://discuss.elastic.co/u/ArturGrzella)\
**Post date:** [March 1, 2021, 1:19pm UTC](https://discuss.elastic.co/t/snapshots-on-nfs4-intermittently-reporting-partial/265800/1 "2021-03-01T13:19:47Z")

</div>

Hi, we have an intermittent issue with snapshots via fs on NFS4.  
Once a day we close the repository for a backup to tape. After this backup has been taken we delete the data and create a new repository and an initial snapshot.  
Then, we are taking snapshots every 15 min. Very few of these snapshots report a PARTIAL status for some 1-3 indices.  
E.g.

```auto
IndexShardSnapshotFailedException[Failed to write commit point]; nested: FileSystemException[/backuprepo/indices/kEmnbUZJSBKVcNFMND95Hw/0/snap-CbacWLlcSw2xagCBh6GxQQ.dat: Input/output error]

```

or

```auto
IndexShardSnapshotFailedException[java.nio.file.DirectoryIteratorException: java.nio.file.FileSystemException: /data/backup-folder/smg-pp-b_backuprepo/indices/pK8Fc_wfS1-yVde-ccwrXw/0: Input/output error]; nested: DirectoryIteratorException[java.nio.file.FileSystemException: /data/backup-folder/smg-pp-b_backuprepo/indices/pK8Fc_wfS1-yVde-ccwrXw/0: Input/output error]; nested: FileSystemException[/backuprepo/indices/pK8Fc_wfS1-yVde-ccwrXw/0: Input/output error]; 

```

We also face other error messages.  
The next snapshot report messages always report SUCCESS.

As I suspect a rare edge-case NFS4 bug being the root cause, my question is:  
If a snapshot reports PARTIAL status and the next ones report SUCCESS, do these successful snapshots _heal_ the affected repositor?. For better understanding: if an index is not snapshotted successfully, does the next successful snapshot fix the issue, so the backed up data is valid and consistent?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [March 1, 2021, 2:42pm UTC](https://discuss.elastic.co/t/snapshots-on-nfs4-intermittently-reporting-partial/265800/2 "2021-03-01T14:42:02Z")

</div>

> [@ArturGrzella](#):
>
> does the next successful snapshot fix the issue

_Technically_ yes, snapshots are logically independent so if a snapshot reports `SUCCESS` then it contains all the data you wanted, regardless of any other failed/partial snapshots. However `Input/output error` indicates something is fundamentally wrong with your storage, and therefore it may not be possible for Elasticsearch to read the data it just successfully wrote. I would not trust this repository with your data.

---

<div class="post-metadata">

**Author:** ![ArturGrzella](https://avatars.discourse-cdn.com/v4/letter/a/b5a626/32.png) [@ArturGrzella](https://discuss.elastic.co/u/ArturGrzella)\
**Post date:** [March 2, 2021, 8:16am UTC](https://discuss.elastic.co/t/snapshots-on-nfs4-intermittently-reporting-partial/265800/3 "2021-03-02T08:16:11Z")

</div>

Thanks @DavidTurner . Your help is very much appreciated.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 30, 2021, 8:16am UTC](https://discuss.elastic.co/t/snapshots-on-nfs4-intermittently-reporting-partial/265800/4 "2021-03-30T08:16:50Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.