# Failed to verify all nodes have repository access

**URL:** <https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467>\
**Category:** Elasticsearch\
**Created:** [March 11, 2022, 3:21pm UTC](https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467 "2022-03-11T15:21:02Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Caio\_Cesar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/caio_cesar/32/102936_2.png) [@Caio\_Cesar](https://discuss.elastic.co/u/Caio_Cesar)\
**Post date:** [March 11, 2022, 3:21pm UTC](https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467/1 "2022-03-11T15:21:02Z")

</div>

hello, I have a problem regarding backup using curator, I use version 6.3.2 of Elasticsearch and curator in version 5.7.6 , backups are stored in a google gsc bucket with all permissions but this error always appears  
Failed to complete action: snapshot. \<class 'curator.exceptions.ActionError'\>: Failed to verify all nodes have repository access: --- Got a 500 response from Elasticsearch. Error message: repository\_verification\_exception  
when creating the snapshots, could you help me?

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 16, 2022, 4:51pm UTC](https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467/2 "2022-03-16T16:51:21Z")

</div>

There are usually good reasons that Elasticsearch should verify the snapshot repository. However, in older versions, like 6.3, not all nodes responded quickly enough to prevent this sort of message.

If you're _completely_ certain there isn't something else going wrong with the repository, there's a way to skip the check:

> **[skip\_repo\_fs\_check | Curator Reference \[5.7\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/client/curator/5.7/option_skip_fsck.html)**

---

<div class="post-metadata">

**Author:** ![Caio\_Cesar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/caio_cesar/32/102936_2.png) [@Caio\_Cesar](https://discuss.elastic.co/u/Caio_Cesar)\
**Post date:** [March 17, 2022, 6:31pm UTC](https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467/3 "2022-03-17T18:31:52Z")

</div>

thanks for your feedback,

I'm storing this in a google gcs bucket, I used this parameter but it still fails

---

<div class="post-metadata">

**Author:** ![Caio\_Cesar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/caio_cesar/32/102936_2.png) [@Caio\_Cesar](https://discuss.elastic.co/u/Caio_Cesar)\
**Post date:** [March 21, 2022, 2:46am UTC](https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467/4 "2022-03-21T02:46:10Z")

</div>

generated this error only on one of the data nodes, the permissions on the bucket are the same as on the other nodes

- missing]","node\_id":"GOuzdEwtQ66KIuBpzqbXNg","status":"INTERNAL\_SERVER\_ERROR"},{"index":"8yace29ju3-metrics-202203","index\_uuid":"8yace29ju3-metrics-202203","shard\_id":5,"reason":"RepositoryMissingException[[backup\_metrics\_sensedia\_prod] missing]","node\_id":"GOuzdEwtQ66KIuBpzqbXNg","status":"INTERNAL\_SERVER\_ERROR"},{"index":"f3k3ojyjhl-metrics-202202","index\_uuid":"f3k3ojyjhl-metrics-202202","shard\_id":3,"reason  
Shards":{"total":366,"failed":45,"successful":321}

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 21, 2022, 2:03pm UTC](https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467/5 "2022-03-21T14:03:48Z")

</div>

Do you have the API call you used to create the repository? It seems that Elasticsearch itself cannot complete the request.

If you dial up Curator's logging to DEBUG level, you will see the full API call made to start the snapshot. If you were to execute that API call using `curl` or some other direct REST method, you would get the same result. It appears that Curator is not the problem if you're getting `INTERNAL_SERVER_ERROR` type messages (which are coming directly from Elasticsearch).

Either the shards are missing/unavailable, or the node in question (`"node_id":"GOuzdEwtQ66KIuBpzqbXNg"`) is unable to contact/reach the repository, or something worse, perhaps. I can't tell from just these messages, but that's just not a good sign.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 18, 2022, 2:03pm UTC](https://discuss.elastic.co/t/failed-to-verify-all-nodes-have-repository-access/299467/6 "2022-04-18T14:03:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
