# Elasticsearch snapshot/restore to s3

**URL:** <https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517>\
**Category:** Elasticsearch\
**Tags:** snapshot-and-restore\
**Created:** [April 21, 2023, 11:35am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517 "2023-04-21T11:35:24Z")\
**Posts on this page:** 12\
**Page:** 2

<div class="post-metadata">

**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\
**Post date:** [April 25, 2023, 5:11am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/21 "2023-04-25T05:11:59Z")

</div>

Thanks leandro, but still issue persists,  
but i see the below error

"[2023-04-25T05:09:51.398+00:00][INFO][plugins.snapshotRestore] You cannot use snapshot\_restore because license information is not available at this time"

---

<div class="post-metadata">

**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\
**Post date:** [April 26, 2023, 11:41am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/22 "2023-04-26T11:41:50Z")

</div>

Hi,

I receive the different error after updating endpoint in elasticsearch.yml

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "repository_exception",
        "reason": "[s3_repository_demo] Could not determine repository generation from root blobs"
      }
    ],
    "type": "repository_exception",
    "reason": "[s3_repository_demo] Could not determine repository generation from root blobs",
    "caused_by": {
      "type": "i_o_exception",
      "reason": "Exception when listing blobs by prefix [index-]",
      "caused_by": {
        "type": "sdk_client_exception",
        "reason": "Unable to execute HTTP request: s3-cn-northwest-1.amazonaws.com",
        "caused_by": {
          "type": "unknown_host_exception",
          "reason": "s3-cn-northwest-1.amazonaws.com"
        }
      }
    }
  },
  "status": 500

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 26, 2023, 11:55am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/23 "2023-04-26T11:55:25Z")

</div>

> [@sraman](#):
>
> `s3-cn-northwest-1.amazonaws.com"`

This is not a valida [s3 endpoint](https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region).

It should be `s3.cn-northwest-1.amazonws.com` according to the `s3` documentation.

But why you change it? I see no reason, your original error is a permissions error, you need to check the permissions of the access key and secret key.

---

<div class="post-metadata">

**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\
**Post date:** [April 26, 2023, 12:14pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/24 "2023-04-26T12:14:31Z")

</div>

Again same error only

```auto
{
  "error": {
    "root_cause": [
      {
        "type": "repository_exception",
        "reason": "[s3_repository_demo] Could not determine repository generation from root blobs"
      }
    ],
    "type": "repository_exception",
    "reason": "[s3_repository_demo] Could not determine repository generation from root blobs",
    "caused_by": {
      "type": "i_o_exception",
      "reason": "Exception when listing blobs by prefix [index-]",
      "caused_by": {
        "type": "sdk_client_exception",
        "reason": "Unable to execute HTTP request: s3.cn-northwest-1.amazonaws.com",
        "caused_by": {
          "type": "unknown_host_exception",
          "reason": "s3.cn-northwest-1.amazonaws.com"
        }
      }
    }
  },
  "status": 500
}

```

Permission works fine inside the ELK server with the same key. Found no issues with keys while accessing s3

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 26, 2023, 12:52pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/25 "2023-04-26T12:52:58Z")

</div>

> [@sraman](#):
>
> Permission works fine inside the ELK server with the same key. Found no issues with keys while accessing s3

Not sure what would be the issue then, it is clearly saying that the permission was wrong before, now it is an error with the endpoint you are using.

I would remove all the `s3` configuration from the `elasticsearch-keystore` and start again.

How are you adding the `access_key` and `secret_key` to the keystore? Are you pasting the values in the terminal?

If you are pasting the values I would recommend that you add it using the `--stdin` option, put the access\_key on a file and cat this file and pipe the result to the elasticsearch-keystore tool as explained in the [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/elasticsearch-keystore.html#add-string-to-keystore).

For example, create a file named `s3-access-key.txt` and put only the access\_key on it.

```auto
your_access_key

```

Then use this command inside `/usr/share/elasticsearch/`

```auto
cat s3-access-key.txt | bin/elasticsearch-keystore add --stdin s3.client.default.access_key

```

Do the same thing to the `secret_key` and test again.

---

<div class="post-metadata">

**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\
**Post date:** [April 27, 2023, 5:54am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/26 "2023-04-27T05:54:23Z")

</div>

No luck leandro, i have removed the s3 configuration from elasticsearch-keystore added the keys using --stdin option as suggested. Now it is permission error as previous

```auto
{
  "name": "ResponseError",
  "meta": {
    "body": {
      "error": {
        "root_cause": [
          {
            "type": "repository_verification_exception",
            "reason": "[s3-elk-repository] path is not accessible on master node"
          }
        ],
        "type": "repository_verification_exception",
        "reason": "[s3-elk-repository] path is not accessible on master node",
        "caused_by": {
          "type": "i_o_exception",
          "reason": "Unable to upload object [tests-khuYNvqJQti4KIPe36s-fA/master.dat] using a single upload",
          "caused_by": {
            "type": "amazon_s3_exception",
            "reason": "The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 7SFDS66X6ZPGMMC3; S3 Extended Request ID: HooZ1crC3nNu0zJs1e3CuV0lj2Ncp7aK6uooq8PGR1/72xWy0HauTA8EPkGu1LyE2Sy8BhO2fnc=; Proxy: null)"
          }
        }
      },
      "status": 500
    },
    "statusCode": 500,
    "headers": {
      "x-opaque-id": "374c5f54-e53e-4808-8293-971a2c752d3e;kibana:application:management:",
      "x-elastic-product": "Elasticsearch",
      "content-type": "application/json;charset=utf-8",
      "content-length": "739"
    },
    "meta": {
      "context": null,
      "request": {
        "params": {
          "method": "POST",
          "path": "/_snapshot/s3-elk-repository/_verify",
          "querystring": "",
          "headers": {
            "user-agent": "Kibana/8.6.2",
            "x-elastic-product-origin": "kibana",
            "authorization": "Basic ZWxhc3RpYzo5QnArX0FmV3ZNc05rTngwNFVKcQ==",
            "x-opaque-id": "374c5f54-e53e-4808-8293-971a2c752d3e;kibana:application:management:",
            "x-elastic-client-meta": "es=8.4.0p,js=16.18.1,t=8.2.0,hc=16.18.1",
            "accept": "application/vnd.elasticsearch+json; compatible-with=8,text/plain"
          }
        },
        "options": {
          "opaqueId": "374c5f54-e53e-4808-8293-971a2c752d3e;kibana:application:management:",
          "headers": {
            "x-elastic-product-origin": "kibana",
            "user-agent": "Kibana/8.6.2",
            "authorization": "Basic ZWxhc3RpYzo5QnArX0FmV3ZNc05rTngwNFVKcQ==",
            "x-opaque-id": "374c5f54-e53e-4808-8293-971a2c752d3e",
            "x-elastic-client-meta": "es=8.4.0p,js=16.18.1,t=8.2.0,hc=16.18.1"
          }
        },
        "id": 1
      },

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [April 27, 2023, 12:43pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/27 "2023-04-27T12:43:27Z")

</div>

How did you create the repository in Elasticsearch?

Also, you created the bucket in s3, right?

For example, if you use the following request to create an repository:

```auto
PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "my-bucket"
  }
}

```

You need to create the bucket `my-bucket` first in AWS first.

Also, check if your access\_key has these [permissions](https://www.elastic.co/guide/en/elasticsearch/reference/8.7/repository-s3.html#repository-s3-permissions).

---

<div class="post-metadata">

**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\
**Post date:** [April 28, 2023, 5:08am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/28 "2023-04-28T05:08:58Z")

</div>

Both ways, i tried via dev tools and snapshot/restore console,  
We have created the s3 bucket first and then repository.

Access key has all the permission.

---

<div class="post-metadata">

**Author:** ![Dasher](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dasher/32/137841_2.png) [@Dasher](https://discuss.elastic.co/u/Dasher)\
**Post date:** [April 28, 2023, 5:17am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/29 "2023-04-28T05:17:11Z")

</div>

Can you try with a new access key with all necessary permission

---

<div class="post-metadata">

**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\
**Post date:** [April 28, 2023, 6:20am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/30 "2023-04-28T06:20:53Z")

</div>

I have tried dasher but no luck.

---

<div class="post-metadata">

**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\
**Post date:** [May 17, 2023, 5:27am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/31 "2023-05-17T05:27:31Z")

</div>

Thanks issue is fixed after adding the endpoint.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 14, 2023, 5:27am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/32 "2023-06-14T05:27:53Z")

</div>

Closed on June 14, 2023, 5:27am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

[Previous page](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517.md?page=1)