# S3 repository Snapshot fails on ES 6.2 **URL:** \ **Category:** Elasticsearch\ **Created:** [March 16, 2018, 1:21pm UTC](https://discuss.elastic.co/t/s3-repository-snapshot-fails-on-es-6-2/124299 "2018-03-16T13:21:14Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![emillykkejensen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emillykkejensen/32/33984_2.png) [@emillykkejensen](https://discuss.elastic.co/u/emillykkejensen)\ **Post date:** [March 16, 2018, 1:21pm UTC](https://discuss.elastic.co/t/s3-repository-snapshot-fails-on-es-6-2/124299/1 "2018-03-16T13:21:15Z")
I’m having trouble creating a S3 snapshot repository for ES 6.2 on DigitalOcean. While the S3 repository plugin works fine with ES 5.6, I can’t get it to work on ES 6.2. Here is what I have done: 1: Installed ES 6.2 with the S3 snapshot repository and reindexed some data over to the new server. 2: Completed the steps here: [https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/repository-s3-client.html](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/repository-s3-client.html) and created AWS access keys in the elasticsearch keystore. 3: Realized that there seems to be a problem with that ([https://github.com/elastic/elasticsearch/issues/27765](https://github.com/elastic/elasticsearch/issues/27765) and [Failed to create S3 repository with ES 6.0](https://discuss.elastic.co/t/failed-to-create-s3-repository-with-es-6-0/108224/7)) and therefor added this line: `-Des.allow_insecure_settings=true` to the file jvm.options in /etc/elasticsearch 4: Still getting an error, every time I try to create the repository The code I try to run: ``` curl -XPUT "http://localhost:9200/_snapshot/snapshot_ESmain" -H "Content-Type: application/json" -d { "type":"s3", "settings":{ "bucket":"mediadata", "region":"AMS3", "endpoint":"ams3.digitaloceanspaces.com", "protocol":"https", "base_path":"snapshot_ESmain", "compress":true, "access_key":"MYNOTSOSECRETKET", "secret_key":"MYSECRETKEY"} } ``` …and the response I get: ``` {"error":{ "root_cause":[{ "type":"blob_store_exception", "reason":"Failed to check if blob [master.dat-temp] exists"}], "type":"blob_store_exception", "reason":"Failed to check if blob [master.dat-temp] exists", "caused_by":{ "type":"amazon_s3_exception", "reason":"Forbidden ( Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: THEREQUESTID; S3 Extended Request ID: THEEXTENDEDREQUESTID)"}}, "status":500} ``` What am I doing wrong? ---
**Author:** ![rjernst](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rjernst/32/6363_2.png) [@rjernst](https://discuss.elastic.co/u/rjernst)\ **Post date:** [March 16, 2018, 5:34pm UTC](https://discuss.elastic.co/t/s3-repository-snapshot-fails-on-es-6-2/124299/2 "2018-03-16T17:34:31Z")
It appears you are running your own s3 clone. Note that we do not explicitly support this. Have you checked logs in your s3 server to see if an error exists on that side? Also note that `region` was removed as parameter (although I might be mixed up on the timeline when it was removed, but I thought it was for 6.0). You should be getting an unknown parameter error for that... Additionally, by specifying your access and secret key inside the keystore, you should not need either the allow\_insecure\_settings sysprop or the access/secret key settings in your request. ---
**Author:** ![emillykkejensen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emillykkejensen/32/33984_2.png) [@emillykkejensen](https://discuss.elastic.co/u/emillykkejensen)\ **Post date:** [March 18, 2018, 4:54pm UTC](https://discuss.elastic.co/t/s3-repository-snapshot-fails-on-es-6-2/124299/3 "2018-03-18T16:54:13Z")
Yes I'm using DigitalOcean Spaces not AWS, but work's fine on ES 5.6 and can't really see why it wouldn’t on 6.2? Had a look at this [https://www.elastic.co/guide/en/elasticsearch/reference/6.2/breaking\_60\_plugins\_changes.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/breaking_60_plugins_changes.html) and can’t really see, if there is anything I should do different from what I’m doing on ES 5.6. Have tried both with allow\_insecure\_settings=true and access\_key/secret\_key in the request as well as without allow\_insecure\_settings and access\_key/secret\_key in the keystore – but gives me the same error response. Unfortunately I don’t know, if I can access the log in DigitalOcean. ---
**Author:** ![emillykkejensen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/emillykkejensen/32/33984_2.png) [@emillykkejensen](https://discuss.elastic.co/u/emillykkejensen)\ **Post date:** [March 21, 2018, 1:07pm UTC](https://discuss.elastic.co/t/s3-repository-snapshot-fails-on-es-6-2/124299/4 "2018-03-21T13:07:33Z")
Is this of any use?? It's from the log: ``` [2018-03-21T13:05:16,263][INFO][o.e.r.RepositoriesService] [MediaData_ESmain] update repository [snapshot] [2018-03-21T13:05:16,409][WARN][r.suppressed] path: /_snapshot/snapshot, params: {pretty=, repository=snapshot} org.elasticsearch.common.blobstore.BlobStoreException: Failed to check if blob [master.dat-temp] exists at org.elasticsearch.repositories.s3.S3BlobContainer.blobExists(S3BlobContainer.java:78) ~[?:?] at org.elasticsearch.repositories.s3.S3BlobContainer.writeBlob(S3BlobContainer.java:99) ~[?:?] at org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:574) ~[elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.repositories.RepositoriesService.verifyRepository(RepositoriesService.java:211) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.repositories.RepositoriesService$VerifyingRegisterRepositoryListener.onResponse(RepositoriesService.java:414) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.repositories.RepositoriesService$VerifyingRegisterRepositoryListener.onResponse(RepositoriesService.java:399) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.AckedClusterStateUpdateTask.onAllNodesAcked(AckedClusterStateUpdateTask.java:64) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.service.MasterService$SafeAckedClusterStateTaskListener.onAllNodesAcked(MasterService.java:526) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.service.MasterService$AckCountDownListener.onNodeAck(MasterService.java:626) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.service.MasterService$DelegetingAckListener.onNodeAck(MasterService.java:566) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.discovery.zen.ZenDiscovery$1.onNewClusterStateProcessed(ZenDiscovery.java:362) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.discovery.zen.PendingClusterStatesQueue.markAsProcessed(PendingClusterStatesQueue.java:177) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.discovery.zen.ZenDiscovery$3.clusterStateProcessed(ZenDiscovery.java:802) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.service.ClusterApplierService$SafeClusterStateTaskListener.clusterStateProcessed(ClusterApplierService.java:540) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:491) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:432) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:161) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:573) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:244) [elasticsearch-6.2.3.jar:6.2.3] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:207) [elasticsearch-6.2.3.jar:6.2.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151] Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 34D7FE81D3A09BCD; S3 Extended Request ID: nrF6AslBHKrRlT4A04YgkBQhFl0wtoTebFNbOr8dHbqcpdP+wyBGcP689THQgckMGKp5SjbhGH8=) at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639) ~[?:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304) ~[?:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056) ~[?:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) ~[?:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) ~[?:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) ~[?:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) ~[?:?] at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) ~[?:?] at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) ~[?:?] at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4247) ~[?:?] at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4194) ~[?:?] at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1253) ~[?:?] at com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1228) ~[?:?] at com.amazonaws.services.s3.AmazonS3Client.doesObjectExist(AmazonS3Client.java:1304) ~[?:?] at org.elasticsearch.repositories.s3.S3BlobContainer.lambda$blobExists$0(S3BlobContainer.java:76) ~[?:?] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_151] at org.elasticsearch.repositories.s3.SocketAccess.doPrivileged(SocketAccess.java:42) ~[?:?] at org.elasticsearch.repositories.s3.S3BlobContainer.blobExists(S3BlobContainer.java:76) ~[?:?] ... 22 more ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 18, 2018, 1:07pm UTC](https://discuss.elastic.co/t/s3-repository-snapshot-fails-on-es-6-2/124299/5 "2018-04-18T13:07:37Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.