# Elasticsearch | S3 Repository and Snapshot Creation Failure

**URL:** <https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442>\
**Category:** Elasticsearch\
**Created:** [March 8, 2019, 7:25am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442 "2019-03-08T07:25:37Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nikesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikesh/32/86579_2.png) [@Nikesh](https://discuss.elastic.co/u/Nikesh)\
**Post date:** [March 8, 2019, 7:25am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/1 "2019-03-08T07:25:37Z")

</div>

Hi,

I am working on Backup and Restore using s3 on my 4 node cluster.  
According to the [documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/6.6/repository-s3-client.html) I first added access\_key and secret\_key using keystore :  
**bin/elasticsearch-keystore add s3.client.default.access\_key**  
**bin/elasticsearch-keystore add s3.client.default.secret\_key**

Next, I tried creating repository using Command  
**PUT [http://11.51.1.121:9998/\_snapshot/my\_s3\_repository](http://11.51.1.121:9998/_snapshot/my_s3_repository)**

```
{
  "type": "s3",
  "settings": {
    "bucket": "bucket_name"
  }
}

```

But this very initial stage is failing and an error is throw i.e.,

```
{
    "error": {
        "root_cause": [
            {
                "type": "amazon_s3_exception",
                "reason": "amazon_s3_exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: AD129AE011109481; S3 Extended Request ID: WOV0JSl3gf+DwUvArwZnEW7D5XvDeRNhjd8TGFJ84JSCTtLfidqDfLv0Uz8s4QiB0hFq0ygxZjc=)"
            }
        ],
        "type": "amazon_s3_exception",
        "reason": "amazon_s3_exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: AD129AE011109481; S3 Extended Request ID: WOV0JSl3gf+DwUvArwZnEW7D5XvDeRNhjd8TGFJ84JSCTtLfidqDfLv0Uz8s4QiB0hFq0ygxZjc=)"
    },
    "status": 500
}

```

I am using Elasticsearch version 6.4.0  
What are the other settings that I am missing, if any?  
Or is there any access issues that I am missing here? If so, how can I know what I am missing?

---

<div class="post-metadata">

**Author:** ![gabriel\_tessier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel_tessier/32/27911_2.png) [@gabriel\_tessier](https://discuss.elastic.co/u/gabriel_tessier)\
**Post date:** [March 8, 2019, 7:51am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/2 "2019-03-08T07:51:51Z")

</div>

Hi,  
did you set the grant on your IAM role for your instance?  
I often mess this step and get this error. 😊

---

<div class="post-metadata">

**Author:** ![Nikesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikesh/32/86579_2.png) [@Nikesh](https://discuss.elastic.co/u/Nikesh)\
**Post date:** [March 8, 2019, 7:54am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/3 "2019-03-08T07:54:52Z")

</div>

Thanks gabriel for the response  
How do you set the grant on my IAM role for my instances?

---

<div class="post-metadata">

**Author:** ![gabriel\_tessier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel_tessier/32/27911_2.png) [@gabriel\_tessier](https://discuss.elastic.co/u/gabriel_tessier)\
**Post date:** [March 8, 2019, 7:58am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/4 "2019-03-08T07:58:23Z")

</div>

First result in google:

> **[AWS IAM--Manage Permissions](https://aws.amazon.com/iam/details/manage-permissions/)**
>
> Permissions let you specify access to AWS resources. Permissions are granted to IAM entities (users, groups, and roles) and by default these entities start with no permissions. In other words, IAM entities can do nothing in AWS until you grant them...

Hope it can help.

More here:

> <https://stackoverflow.com/questions/9073755/basic-aws-iam-permissions-for-an-s3-bucket>

---

<div class="post-metadata">

**Author:** ![Nikesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikesh/32/86579_2.png) [@Nikesh](https://discuss.elastic.co/u/Nikesh)\
**Post date:** [March 8, 2019, 7:59am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/5 "2019-03-08T07:59:56Z")

</div>

Yes, It sure helps Gabriel!  
Could you please tell me what are the ideal Permissions that has to be provided?

---

<div class="post-metadata">

**Author:** ![gabriel\_tessier](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gabriel_tessier/32/27911_2.png) [@gabriel\_tessier](https://discuss.elastic.co/u/gabriel_tessier)\
**Post date:** [March 8, 2019, 8:05am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/6 "2019-03-08T08:05:39Z")

</div>

Here the resource I used to configure mine:

> **[Elasticsearch Backup : Snapshot and Restore on AWS S3](https://medium.com/@federicopanini/elasticsearch-backup-snapshot-and-restore-on-aws-s3-f1fc32fbca7f)**
>
> how to make backup for Elasticsearch and restore them

---

<div class="post-metadata">

**Author:** ![Nikesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikesh/32/86579_2.png) [@Nikesh](https://discuss.elastic.co/u/Nikesh)\
**Post date:** [March 8, 2019, 9:08am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/7 "2019-03-08T09:08:47Z")

</div>

Thanks for sharing these links Gabriel  
I am not able to download aws-cloud plugin of Elasticsearch.  
It says unknown plugin cloud-aws  
I am currently using Elasticsearch version 6.4.0  
Has these settings differ with my version?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [March 8, 2019, 7:27pm UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/8 "2019-03-08T19:27:59Z")

</div>

The blog post linked above is rather old, and that plugin no longer exists. The [reference manual](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3.html) describes how to do this in more recent versions of Elasticsearch.

---

<div class="post-metadata">

**Author:** ![Nikesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikesh/32/86579_2.png) [@Nikesh](https://discuss.elastic.co/u/Nikesh)\
**Post date:** [March 11, 2019, 5:53am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/9 "2019-03-11T05:53:35Z")

</div>

Thanks David for your response.

> [@Nikesh](#):
>
> I am working on Backup and Restore using s3 on my 4 node cluster.  
> According to the [documentation](https://www.elastic.co/guide/en/elasticsearch/plugins/6.6/repository-s3-client.html) I first added access\_key and secret\_key using keystore :  
> **bin/elasticsearch-keystore add s3.client.default.access\_key**  
> **bin/elasticsearch-keystore add s3.client.default.secret\_key**

I indeed used the same documentation as I mentioned in my question.  
Let me tell you the process I employed.

1. Installed S3 repository plugin using : **sudo bin/elasticsearch-plugin install repository-s3**
2. Added AWS access key settings to the keystore.  
**bin/elasticsearch-keystore add s3.client.default.access\_key**  
**bin/elasticsearch-keystore add s3.client.default.secret\_key**
3. Created a repository using the following  
PUT \_snapshot/my\_s3\_repository

> {  
> "type": "s3",  
> "settings": {  
> "bucket": "my\_bucket"  
> }  
> }

This is providing me the error,

> {  
> "error": {  
> "root\_cause": [  
> {  
> "type": "amazon\_s3\_exception",  
> "reason": "amazon\_s3\_exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: AD129AE011109481; S3 Extended Request ID: WOV0JSl3gf+DwUvArwZnEW7D5XvDeRNhjd8TGFJ84JSCTtLfidqDfLv0Uz8s4QiB0hFq0ygxZjc=)"  
> }  
> ],  
> "type": "amazon\_s3\_exception",  
> "reason": "amazon\_s3\_exception: Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: AD129AE011109481; S3 Extended Request ID: WOV0JSl3gf+DwUvArwZnEW7D5XvDeRNhjd8TGFJ84JSCTtLfidqDfLv0Uz8s4QiB0hFq0ygxZjc=)"  
> },  
> "status": 500  
> }

Please tell me what Am I missing here?

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [March 11, 2019, 9:06am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/10 "2019-03-11T09:06:08Z")

</div>

> [@Nikesh](#):
>
> `"bucket": "my_bucket"`

Is `my_bucket` the name of your bucket?

---

<div class="post-metadata">

**Author:** ![Nikesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikesh/32/86579_2.png) [@Nikesh](https://discuss.elastic.co/u/Nikesh)\
**Post date:** [March 11, 2019, 10:07am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/11 "2019-03-11T10:07:20Z")

</div>

No.  
This is just an example. I have my actual bucket name mentioned under it

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [March 11, 2019, 11:43am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/12 "2019-03-11T11:43:31Z")

</div>

Have you configured your IAM permissions [as per the manual](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-repository.html#repository-s3-permissions)?

---

<div class="post-metadata">

**Author:** ![Nikesh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nikesh/32/86579_2.png) [@Nikesh](https://discuss.elastic.co/u/Nikesh)\
**Post date:** [March 12, 2019, 5:50am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/13 "2019-03-12T05:50:18Z")

</div>

Thanks David for the response.  
My whole problem lied in IAM Permissions  
Now the Backup and restore process is working fine.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 9, 2019, 5:50am UTC](https://discuss.elastic.co/t/elasticsearch-s3-repository-and-snapshot-creation-failure/171442/14 "2019-04-09T05:50:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.