# Repository\_verification\_exception

**URL:** <https://discuss.elastic.co/t/repository-verification-exception/363174>\
**Category:** Elasticsearch\
**Tags:** snapshot-and-restore\
**Created:** [July 15, 2024, 4:39pm UTC](https://discuss.elastic.co/t/repository-verification-exception/363174 "2024-07-15T16:39:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Naresh\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/naresh_kumar1/32/124342_2.png) [@Naresh\_Kumar1](https://discuss.elastic.co/u/Naresh_Kumar1)\
**Post date:** [July 15, 2024, 4:39pm UTC](https://discuss.elastic.co/t/repository-verification-exception/363174/1 "2024-07-15T16:39:07Z")

</div>

I am using ES 8.13.4 in the EKS cluster, On the daily snapshot we are getting snapshot exceptions like connection pool shutdown. What was the reason behind this it was working fine suddenly without any changes we are getting this error.

##Error1:

```auto
{
  "type": "repository_exception",
  "reason": "[daily-snapshot] Could not determine repository generation from root blobs",
  "caused_by": {
    "type": "i_o_exception",
    "reason": "Exception when listing blobs by prefix [index-]",
    "caused_by": {
      "type": "amazon_s3_exception",
      "reason": "Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 3VKADEXMMG4YS8NK; S3 Extended Request ID: 7CLwH5Smt+rWsemLKOm64+2hVVDCPgLp4g2FQT7DlGr/C9n685VKdj+6VMylt2NGwedy4=; Proxy: null)"
    }
  }
}

```

##Erro2

```auto
{
  "type": "snapshot_exception",
  "reason": "[daily-snapshot:vector-2024.07.15-2ktda621quq2miiyynagsq/_y97vgCARAWWUNQNfyECWQ] failed to update snapshot in repository",
  "caused_by": {
    "type": "i_o_exception",
    "reason": "Unable to upload object [meta-_y97vgCARAWWECWQ.dat] using a single upload",
    "caused_by": {
      "type": "amazon_client_exception",
      "reason": "java.lang.IllegalStateException: Connection pool shut down",
      "caused_by": {
        "type": "illegal_state_exception",
        "reason": "Connection pool shut down"
      }
    },
    "suppressed": [
      {
        "type": "i_o_exception",
        "reason": "Unable to upload object [snap-_y97vgCARAWWUNQCWQ.dat] using a single upload",
        "caused_by": {
          "type": "amazon_client_exception",
          "reason": "java.lang.IllegalStateException: Connection pool shut down",
          "caused_by": {
            "type": "illegal_state_exception",
            "reason": "Connection pool shut down"
          }
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [July 16, 2024, 12:44am UTC](https://discuss.elastic.co/t/repository-verification-exception/363174/2 "2024-07-16T00:44:10Z")

</div>

> [@Naresh\_Kumar1](#):
>
> What was the reason behind this it was working fine suddenly without any changes we are getting this error

It looks like the access key that you're using no longer has permission to access that bucket in S3.  
Perhaps someone revoked the key, or changed the IAM policy.

---

<div class="post-metadata">

**Author:** ![Naresh\_Kumar1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/naresh_kumar1/32/124342_2.png) [@Naresh\_Kumar1](https://discuss.elastic.co/u/Naresh_Kumar1)\
**Post date:** [July 16, 2024, 5:08am UTC](https://discuss.elastic.co/t/repository-verification-exception/363174/3 "2024-07-16T05:08:52Z")

</div>

We have not changed anything. earlier it was working fine, and our IAM role and policy has access to the bucket, but we have a condition like session based IAM roles, we have decoded the session also it was fine, but the the nodes not able to access it.

---

<div class="post-metadata">

**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\
**Post date:** [July 16, 2024, 5:28am UTC](https://discuss.elastic.co/t/repository-verification-exception/363174/4 "2024-07-16T05:28:57Z")

</div>

> [@Naresh\_Kumar1](#):
>
> ` "reason": "Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 3VKADEXMMG4YS8NK; S3 Extended Request ID: 7CLwH5Smt+rWsemLKOm64+2hVVDCPgLp4g2FQT7DlGr/C9n685VKdj+6VMylt2NGwedy4=; Proxy: null)"`

This error says otherwise.  
If that error still occurs when you attempt to verify the repository then it's a very strong indication that the access is not what you think it is.  
I think you're going to need to diagnose this from the the AWS/IAM side.