# How can I restore a snapshot from S3 to a different cluster? **URL:** \ **Category:** Elasticsearch\ **Tags:** snapshot-and-restore\ **Created:** [October 3, 2024, 10:06am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182 "2024-10-03T10:06:49Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![summoner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/summoner/32/138051_2.png) [@summoner](https://discuss.elastic.co/u/summoner)\ **Post date:** [October 3, 2024, 10:06am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/1 "2024-10-03T10:06:49Z")
I have managed to get a snapshot stored on S3, now trying to get that snapshot onto my elasticsearch deployment using an operator on GKE. Currently I have 1 master data pod `elasticsearch-config-es-master-data-1` active. When trying to set up an S3 repository, below is my PUT request followed by the error I see. ```auto PUT _snapshot/s3_repo { "type": "s3", "settings": { "bucket": "opensearch-snapshots" } } ``` ERROR: ```auto { "error": { "root_cause": [ { "type": "repository_verification_exception", "reason": "[s3_repo] path is not accessible on master node" } ], "type": "repository_verification_exception", "reason": "[s3_repo] path is not accessible on master node", "caused_by": { "type": "i_o_exception", "reason": "Unable to upload object [tests-FtGw-UFkSSC4NN_DcfxsgQ/master.dat] using a single upload", "caused_by": { "type": "sdk_client_exception", "reason": "The requested metadata is not found at http://[IP address]/latest/meta-data/iam/security-credentials/" } } }, "status": 500 } ``` I'm wondering what I'm missing, any help/guidance is greatly appreciated! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [October 3, 2024, 10:06am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/2 "2024-10-03T10:06:49Z")
OpenSearch/OpenDistro are AWS run products and differ from the original Elasticsearch and Kibana products that Elastic builds and maintains. You may need to contact them directly for further assistance. See [What is OpenSearch and the OpenSearch Dashboard? | Elastic](https://www.elastic.co/elasticsearch/opensearch) for more details. (This is an automated response from your friendly Elastic bot. Please report this post if you have any suggestions or concerns :elasticheart: ) ---
**Author:** ![summoner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/summoner/32/138051_2.png) [@summoner](https://discuss.elastic.co/u/summoner)\ **Post date:** [October 3, 2024, 10:07am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/3 "2024-10-03T10:07:25Z")
Removed #opensearch ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 3, 2024, 10:12am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/4 "2024-10-03T10:12:39Z")
> [@summoner](#): > > `"bucket": "opensearch-snapshots"` OpenSearch is a different product from Elasticsearch (with some shared heritage ofc) and their snapshot formats are not compatible. If these snapshots were created with OpenSearch then Elasticsearch isn't going to be able to read them. > [@summoner](#): > > `"reason": "The requested metadata is not found at http://[IP address]/latest/meta-data/iam/security-credentials/"` This indicates that you haven't configured the `access_key` and `secret_key` so Elasticsearch is trying to use the EC2 IMDS to obtain the credentials it needs to access the bucket, but IMDS doesn't have any credentials to offer. ---
**Author:** ![summoner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/summoner/32/138051_2.png) [@summoner](https://discuss.elastic.co/u/summoner)\ **Post date:** [October 3, 2024, 11:10am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/5 "2024-10-03T11:10:11Z")
Thanks for the response, David 🙏🏽 > If these snapshots were created with OpenSearch then Elasticsearch isn't going to be able to read them. I created the snapshot by running a PUT request to the OpenSearch domain `https://[OpenSearch domain name].us-west-2.es.amazonaws.com/_snapshot/test-snapshot-repo/test-snapshot`, is this considered as 'a snapshot created with OpenSearch'? Just wanted a clarification. > This indicates that you haven't configured the `access_key` and `secret_key` I've tried multiple ways to configure this 1. Setting as environment variable in my deployment pod `elasticsearch-config-es-master-data-1`. 2. Adding them in elasticsearch keystore ```auto bin/elasticsearch-keystore add s3.client.default.access_key bin/elasticsearch-keystore add s3.client.default.secret_key bin/elasticsearch-keystore add s3.client.default.session_token ``` 1. On the PUT request (I'm using Postman), I have included those 3 fields in the authorization as well. Are these approaches incorrect? ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 3, 2024, 11:23am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/6 "2024-10-03T11:23:18Z")
> [@summoner](#): > > is this considered as 'a snapshot created with OpenSearch'? Yes. > [@summoner](#): > > Setting as environment variable That won't do anything. > [@summoner](#): > > I have included those 3 fields in the authorization That also won't do anything. > [@summoner](#): > > Adding them in elasticsearch keystore That should work, but it does require that you're using the `default` S3 client and that `bin/elasticsearch-keystore` is running in the same environment as you launched ES (in particular, it has the same value for `${ES_PATH_CONF}`). ---
**Author:** ![summoner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/summoner/32/138051_2.png) [@summoner](https://discuss.elastic.co/u/summoner)\ **Post date:** [October 3, 2024, 12:34pm UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/7 "2024-10-03T12:34:12Z")
Oh no... So, for people (sadly me) who want to migrate their team away from OpenSearch to manage their own using elasticsearch operator, what are my alternatives so that data is preserved? ☹ > That should work, but it does require that you're using the `default` S3 client Is this referring to the repository I've created that is on my destination cluster? How can I check this? > `bin/elasticsearch-keystore` is running in the same environment as you launched ES hmm, the operator I'm using deploys these pods: ```auto elastic-operator-klasjhdakslgjh elasticsearch-config-es-master-data-1 kibana-config-kb-wooshdhf-akdfjg ``` I've been making configurations in `elasticsearch-config-es-master-data-1` since the pod has elasticsearch and all its other commands in the `bin/` directory. So, I'm hoping at least this part is correct. (thanks again for answering my questions) ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [October 3, 2024, 2:27pm UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/8 "2024-10-03T14:27:27Z")
> [@summoner](#): > > So, for people (sadly me) who want to migrate their team away from OpenSearch to manage their own using elasticsearch operator, what are my alternatives so that data is preserved? ☹ I think you could find a lot of ideas in this blog post: > **[Migrating 1 billion log lines from OpenSearch to Elasticsearch — Elastic...](https://www.elastic.co/observability-labs/blog/migrating-billion-log-lines-opensearch-elasticsearch)** > > Learn how to migrate 1 billion log lines from OpenSearch to Elasticsearch for improved performance and reduced disk usage. Discover the migration strategies, data transfer methods, and optimization techniques used in this guide. ---
**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\ **Post date:** [October 3, 2024, 5:45pm UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/9 "2024-10-03T17:45:47Z")
> [@summoner](#): > > Is this referring to the repository I've created that is on my destination cluster? How can I check this? The client is the one you choose in the `PUT /_snapshot/test-snapshot-repo` call. By default it's `default`. See the docs for more info: > **[S3 repository | Elasticsearch Guide \[8.15\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/repository-s3.html)** ---
**Author:** ![summoner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/summoner/32/138051_2.png) [@summoner](https://discuss.elastic.co/u/summoner)\ **Post date:** [October 4, 2024, 3:58am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/10 "2024-10-04T03:58:08Z")
Thank you for this resource, David. Seems like a lengthy process but at least I have a starting point and can kinda get an idea what I need to do 😃 ---
**Author:** ![summoner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/summoner/32/138051_2.png) [@summoner](https://discuss.elastic.co/u/summoner)\ **Post date:** [October 4, 2024, 3:59am UTC](https://discuss.elastic.co/t/how-can-i-restore-a-snapshot-from-s3-to-a-different-cluster/368182/11 "2024-10-04T03:59:24Z")
Ok, my configuration is correct then! Thanks again for the information and help! 😃