# Elasticsearch snapshot/restore to s3 **URL:** \ **Category:** Elasticsearch\ **Tags:** snapshot-and-restore\ **Created:** [April 21, 2023, 11:35am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517 "2023-04-21T11:35:24Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 21, 2023, 11:35am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/1 "2023-04-21T11:35:24Z")
Hi, I have installed elasticsearch 8.6.2 & kibana 8.6.2 on the same standalone server for testing purpose. Planning to place the data snapshot to S3 and restore, but facing issues while creating the repository(it's not getting connected). From server i been able to communicate with S3 but not with kibana UI. I receive the following error during verification status. Can someone help on this ```auto "name": "ResponseError", "meta": { "body": { "error": { "root_cause": [ { "type": "repository_verification_exception", "reason": "[****] path is not accessible on master node" } ], "type": "repository_verification_exception", "reason": "[****] path is not accessible on master node", "caused_by": { "type": "i_o_exception", "reason": "Unable to upload object [tests-2cIEc1oJRoabyhojn98MQg/master.dat] using a single upload", "caused_by": { "type": "amazon_s3_exception", "reason": "The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: NSHPXN230XB9DBTR; S3 Extended Request ID: G15vrRvWfJuf/OSnqEjw9lA/o7tV48ac9ICgWz6yfcD703akbi/zIeneVJ6vM+OrHV19+wLhVvg=; Proxy: null)" } } }, "status": 500 }, "statusCode": 500, "headers": { "x-opaque-id": "47b9fb6e-9077-4ddf-ba5c-2aaf6c85db3e;kibana:application:management:", "x-elastic-product": "Elasticsearch", "content-type": "application/json;charset=utf-8", "content-length": "721" }, "meta": { "context": null, "request": { "params": { "method": "POST", "path": "/_snapshot/ **** /_verify", "querystring": "", "headers": { "user-agent": "Kibana/8.6.2", "x-elastic-product-origin": "kibana", "authorization": "Basic ZWxhc3RpYzo5QnArX0FmV3ZNc05rTngwNFVKcQ==", "x-opaque-id": "47b9fb6e-9077-4ddf-ba5c-2aaf6c85db3e;kibana:application:management:", "x-elastic-client-meta": "es=8.4.0p,js=16.18.1,t=8.2.0,hc=16.18.1", "accept": "application/vnd.elasticsearch+json; compatible-with=8,text/plain" } }, "options": { "opaqueId": "47b9fb6e-9077-4ddf-ba5c-2aaf6c85db3e;kibana:application:management:", "headers": { "x-elastic-product-origin": "kibana", "user-agent": "Kibana/8.6.2", "authorization": "Basic ZWxhc3RpYzo5QnArX0FmV3ZNc05rTngwNFVKcQ==", "x-opaque-id": "47b9fb6e-9077-4ddf-ba5c-2aaf6c85db3e", "x-elastic-client-meta": "es=8.4.0p,js=16.18.1,t=8.2.0,hc=16.18.1" } }, "id": 1 }, ``` Thanks, Seetharaman ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [April 21, 2023, 1:15pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/2 "2023-04-21T13:15:54Z")
> [@sraman](#): > > From server i been able to communicate with S3 How did you test this? Kibana is UI for Elasticsearch, it does not talk with S3, it is your node that will talk to it, your error is not a communication error, but a permissions error. Check the message: > The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: **InvalidAccessKeyId** ; Request ID: NSHPXN230XB9DBTR; S3 Extended Request ID: G15vrRvWfJuf/OSnqEjw9lA/o7tV48ac9ICgWz6yfcD703akbi/zIeneVJ6vM+OrHV19+wLhVvg=; Proxy: null You need to double check the permissions you used while creating the repository. ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 21, 2023, 1:53pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/3 "2023-04-21T13:53:48Z")
Hi leandrojmp, I have tested from node which holds both elasticsearch n kibana using aws cli.. Node able to communicate with s3 and also uploaded objects to specified bucket, but UI says its a invalid key n permission Where's node doesn't. Thanks, Seetharaman ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 21, 2023, 1:56pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/4 "2023-04-21T13:56:44Z")
Also made the bucket to public which in turn returns the same error. ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [April 21, 2023, 2:08pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/5 "2023-04-21T14:08:42Z")
How did you add the credentials in Elasticsearch? According to this [documentation](https://www.elastic.co/guide/en/elasticsearch/reference/8.6/repository-s3.html)? Did you restarted the node or reloaded the secure settings after adding the credentials? ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 21, 2023, 2:38pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/6 "2023-04-21T14:38:39Z")
Yes i have added the keys both in elasticsearch keystore and aws configure and also restarted all the services and node. How to reload secure settings? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 21, 2023, 2:58pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/7 "2023-04-21T14:58:54Z")
If you restarted the nodes, that should be ok. ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 21, 2023, 3:16pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/8 "2023-04-21T15:16:04Z")
Restart doesn't work david ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 21, 2023, 4:35pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/9 "2023-04-21T16:35:03Z")
You need to share more I think: - What exact command did you ran to update your credentials? - How did you define your repository? May be share the output of: ```auto GET /_snapshot ``` ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 21, 2023, 5:21pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/10 "2023-04-21T17:21:58Z")
> What exact command did you ran to update your credentials? > /usr/share/elasticsearch/bin/.elasticsearch-keystore add keys > GET /\_snapshot ```auto { "elk-demo": { "type": "s3", "settings": { "bucket": "elk-cn-registry" } } } ``` > POST /\_snapshot/elk-demo/\_verify ```auto { "error": { "root_cause": [ { "type": "repository_verification_exception", "reason": "[elk-demo] path is not accessible on master node" } ], "type": "repository_verification_exception", "reason": "[elk-demo] path is not accessible on master node", "caused_by": { "type": "i_o_exception", "reason": "Unable to upload object [tests-amh18k19S_KlnGGGvMfcuA/master.dat] using a single upload", "caused_by": { "type": "amazon_s3_exception", "reason": "The AWS Access Key Id you provided does not exist in our records. (Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: QATQ4EDR2ZTEE5HK; S3 Extended Request ID: i7fJupZH2EbZ/KYZrnjBLpG//3JRrK6BAXmC6wn2s4Rlw/YXkv5yDcAn+Zmns1QEFtQuXxzR5ME=; Proxy: null)" } } }, "status": 500 } ``` ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [April 21, 2023, 5:30pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/11 "2023-04-21T17:30:10Z")
> [@sraman](#): > > /usr/share/elasticsearch/bin/.elasticsearch-keystore add keys Which commands exactly did you run? You need to run 2 commands. One to add the access\_key ```auto bin/elasticsearch-keystore add s3.client.default.access_key ``` And one to add the secret\_key ```auto bin/elasticsearch-keystore add s3.client.default.secret_key ``` ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 22, 2023, 2:42am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/12 "2023-04-22T02:42:38Z")
I have added both the keys access n and secret to the respective keystore path ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 22, 2023, 7:17am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/13 "2023-04-22T07:17:45Z")
Could you run this: ``` bin/elasticsearch-keystore list ``` And share here the output without touching anything? ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 22, 2023, 8:14am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/14 "2023-04-22T08:14:51Z")
sure, please find the output below ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/5/35aee8210a8b6a05e9a087fbb933805797da7259.png) ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 22, 2023, 8:42am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/15 "2023-04-22T08:42:44Z")
Did you do that on all nodes? ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 22, 2023, 8:58am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/16 "2023-04-22T08:58:35Z")
Just one node david.. does metrics server needs this configurations??? We are collecting only logs n metrics from that server. ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [April 22, 2023, 9:16am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/17 "2023-04-22T09:16:37Z")
What's the relationship with logs and metrics when you are talking first of doing backups with s3 repositories ? I meant that every node in the cluster needs those settings. ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 24, 2023, 6:45am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/18 "2023-04-24T06:45:33Z")
Hi David, We have only one node that has Elasticsearch & kibana and i have added the keys in that node. Another machine is a client, we have just installed the filebeat,metricbeat for monitoring purpose. ---
**Author:** ![sraman](https://avatars.discourse-cdn.com/v4/letter/s/e47774/32.png) [@sraman](https://discuss.elastic.co/u/sraman)\ **Post date:** [April 24, 2023, 10:00am UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/19 "2023-04-24T10:00:13Z")
One more question, does basic license involved snapshot/restore feature? ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [April 24, 2023, 12:01pm UTC](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517/20 "2023-04-24T12:01:17Z")
> [@sraman](#): > > We have only one node that has Elasticsearch & kibana and i have added the keys in that node. > Another machine is a client, we have just installed the filebeat,metricbeat for monitoring purpose. If you added the keys and restarted the node, there is not much to troubleshoot, the error is pretty key the credentials are not working. I would remove the setting from the keystore and add them again double checking the values. > [@sraman](#): > > One more question, does basic license involved snapshot/restore feature? Yes, snapshot and restore works with the basica license. [Next page](https://discuss.elastic.co/t/elasticsearch-snapshot-restore-to-s3/330517.md?page=2)