# Getting repository verification exception

**URL:** <https://discuss.elastic.co/t/getting-repository-verification-exception/234811>\
**Category:** Elasticsearch\
**Created:** [May 28, 2020, 7:55pm UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811 "2020-05-28T19:55:41Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![hello5346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello5346/32/68209_2.png) [@hello5346](https://discuss.elastic.co/u/hello5346)\
**Post date:** [May 28, 2020, 7:55pm UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811/1 "2020-05-28T19:55:41Z")

</div>

I am trying to create snapshots of my ealsticsearch in s3 bucket. i had created an s3 bucket with public access and created an user with the following permission

```
       {
"Statement": [
    {
        "Action": [
            "s3:ListBucket",
            "s3:GetBucketLocation",
            "s3:ListBucketMultipartUploads",
            "s3:ListBucketVersions"
        ],
        "Effect": "Allow",
        "Resource": [
            "arn:aws:s3:::elastic-snapshots"
        ]
    },
    {
        "Action": [
            "s3:GetObject",
            "s3:PutObject",
            "s3:AbortMultipartUpload",
            "s3:ListMultipartUploadParts"
        ],
        "Effect": "Allow",
        "Resource": [
            "arn:aws:s3:::elastic-snapshots",
            "arn:aws:s3:::elastic-snapshots/*"
        ]
    }
],
"Version": "2012-10-17"

```

}

after that installed s3 repository plugin by using this

```
sudo bin/elasticsearch-plugin install repository-s3

```

and added secret-key and access key by following commands

```
   bin/elasticsearch-keystore add s3.client.default.access_key
   bin/elasticsearch-keystore add s3.client.default.secret_key

```

and tried to create snapshot\_repository with following command

```
curl -X PUT "10.0.0.198:9200/_snapshot/snapshot_repository?pretty" -H 'Content-Type: application/json' -d'{ "type": "s3", "settings": { "region": "us-west-1",
"bucket": "elastic-snapshots", "base_path": "/snapshot-repository/",
"endpoint": "s3-us-west-1.amazon.aws.com" } }'

```

it is giving me the following exception

```
     "error" : {
"root_cause" : [
  {
    "type" : "repository_verification_exception",
    "reason" : "[snapshot_repository] path [/snapshot-repository/] is not accessible on master node"
  }
],
"type" : "repository_verification_exception",
"reason" : "[snapshot_repository] path [/snapshot-repository/] is not accessible on master node",
"caused_by" : {
  "type" : "i_o_exception",
  "reason" : "Unable to upload object [/snapshot-repository//tests-72t6ssc0TC2oQXObyby9yQ/master.dat] using a single upload",
  "caused_by" : {
    "type" : "sdk_client_exception",
    "reason" : "sdk_client_exception: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/"
  }
}
},
     "status" : 500
    }

```

any help would be appreciated. i'm literally struck here.

---

<div class="post-metadata">

**Author:** ![Armin\_Braun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/armin_braun/32/20092_2.png) [@Armin\_Braun](https://discuss.elastic.co/u/Armin_Braun)\
**Post date:** [May 29, 2020, 12:50pm UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811/2 "2020-05-29T12:50:21Z")

</div>

Hi @hello5346

It looks like the `default` client settings are not getting picked up and that's why the logic is falling back to the EC2 metadata credential resolver which is failing here.

After adding the default credentials to the keystore, did you restart your nodes or reload the secure settings via the [API](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-reload-secure-settings.html)? I would expect either a restart or calling that API to fix things and allow you to add+verify the repository.

---

<div class="post-metadata">

**Author:** ![hello5346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello5346/32/68209_2.png) [@hello5346](https://discuss.elastic.co/u/hello5346)\
**Post date:** [May 29, 2020, 2:40pm UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811/3 "2020-05-29T14:40:46Z")

</div>

Hello @Armin_Braun, Thanks for the answer here is what i had done and my result

I had restarted the system and started elasticsearch and ran this

```
    curl -X PUT "10.0.0.198:9200/_snapshot/s3_repository?pretty" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "elastic-snapshot"
  }
}
' 

```

and the output is still same

> ```
> {
> "error" : {
> "root_cause" : [
> {
> "type" : "repository_verification_exception",
> "reason" : "[s3_repository] path is not accessible on master node"
> }
> ],
> "type" : "repository_verification_exception",
> "reason" : "[s3_repository] path is not accessible on master node",
> "caused_by" : {
> "type" : "i_o_exception",
> "reason" : "Unable to upload object [tests-orwYhYx0ShiMyd6EFC7DIQ/master.dat] using a single upload",
> "caused_by" : {
> "type" : "sdk_client_exception",
> "reason" : "sdk_client_exception: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/"
> }
> }
> },
> "status" : 500
> }
> 
> ```

expecting a reply from you.

---

<div class="post-metadata">

**Author:** ![Armin\_Braun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/armin_braun/32/20092_2.png) [@Armin\_Braun](https://discuss.elastic.co/u/Armin_Braun)\
**Post date:** [May 29, 2020, 3:00pm UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811/4 "2020-05-29T15:00:39Z")

</div>

@hello5346  
could you share a few more details on your setup please:

What version of Elasticsearch are you using?  
Did you make sure to run

```auto
   bin/elasticsearch-keystore add s3.client.default.access_key
   bin/elasticsearch-keystore add s3.client.default.secret_key

```

on all of your cluster nodes?

---

<div class="post-metadata">

**Author:** ![hello5346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello5346/32/68209_2.png) [@hello5346](https://discuss.elastic.co/u/hello5346)\
**Post date:** [May 30, 2020, 9:29am UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811/5 "2020-05-30T09:29:54Z")

</div>

@ [Armin\_Braun](https://discuss.elastic.co/u/armin_braun)  
i'm using 7.6 elasticsearch with 3 node cluster with xpack enabled

```
 bin/elasticsearch-keystore add s3.client.default.access_key
 bin/elasticsearch-keystore add s3.client.default.secret_key

```

I ran the above commands and provided both the keys and after that i had restarted the machine and services as well. I can manually upload the data folder of elasticsearch into the s3 bucket using aws-cli but i want this snapshot-repository to be in place.

I tried this out again after providing the keys:

```
    curl --user elastic:elastic -X PUT "10.0.0.198:9200/_snapshot/snapshot_repository?pretty" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "region": "us-west-1",
    "bucket": "elastic-snapshot",
    "endpoint": "s3-us-west-1.amazon.aws.com"
  }
}
'

{
  "error" : {
"root_cause" : [
  {
    "type" : "repository_verification_exception",
    "reason" : "[snapshot_repository] path is not accessible on master node"
  }
],
"type" : "repository_verification_exception",
"reason" : "[snapshot_repository] path is not accessible on master node",
"caused_by" : {
  "type" : "i_o_exception",
  "reason" : "Unable to upload object [tests-WNnM6viCRb-sgs-YYfBPfQ/master.dat] using a single upload",
  "caused_by" : {
    "type" : "sdk_client_exception",
    "reason" : "sdk_client_exception: The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/"
  }
}
  },
  "status" : 500
}

```

can you identify anymore steps i'm missing.

---

<div class="post-metadata">

**Author:** ![hello5346](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hello5346/32/68209_2.png) [@hello5346](https://discuss.elastic.co/u/hello5346)\
**Post date:** [May 31, 2020, 2:03pm UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811/6 "2020-05-31T14:03:28Z")

</div>

Thanks @[Armin\_Braun](https://discuss.elastic.co/u/armin_braun)  
This has worked for me.

added s3-repository plugin in all the nodes by using this

`bin/elasticsearch-plugin install repository-s3`

after that provided the access-key and secret\_key

```
bin/elasticsearch-keystore add s3.client.default.access_key
bin/elasticsearch-keystore add s3.client.default.secret_key

```

restarted the machine's and services and after that created the snapshot with the following command

```
curl --user elastic:elastic -X PUT "10.0.0.6:9200/_snapshot/my_s3_repository?pretty" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "elastic-snapshot"
  }
}
'
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 28, 2020, 2:03pm UTC](https://discuss.elastic.co/t/getting-repository-verification-exception/234811/7 "2020-06-28T14:03:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.