# Is there a bug in elasticsearch 6.5.0 concerning s3 snapshot repositories

**URL:** <https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250>\
**Category:** Elasticsearch\
**Created:** [August 25, 2020, 8:21am UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250 "2020-08-25T08:21:06Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dimitris\_sb](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@dimitris\_sb](https://discuss.elastic.co/u/dimitris_sb)\
**Post date:** [August 25, 2020, 8:21am UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250/1 "2020-08-25T08:21:06Z")

</div>

Hi All,

This is a single node cluster installation running elasticsearch 6.5.0.  
I noticed that after the set up of the aws access key and secret key in the keystore

P.S. I have tested my access key and secret key with cyberduck and I can connect, read, write objects, etc. in the s3 bucket.

**Request** :

```
PUT _snapshot/es5_aws_test_backup
{
  "type": "s3",
  "settings": {
    "bucket": "metrika-aws-backups"
  }
}

```

**Response** :

```
{
      "error": {
        "root_cause": [
          {
            "type": "repository_exception",
            "reason": "[es5_aws_test_backup] cannot create blob store"
          }
        ],
        "type": "repository_exception",
        "reason": "[es5_aws_test_backup] cannot create blob store",
        "caused_by": {
          "type": "illegal_argument_exception",
          "reason": "you do not have permissions to access the bucket [test-aws-backups]",
          "caused_by": {
            "type": "amazon_s3_exception",
            "reason": "Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: EA157F3D99CF2FF1; S3 Extended Request ID: YpJeteKUoY6I5vSTp7gEVVa1DXjwbHFVIhhYJ80/F2Vu2kIHKN/wfreoGhNZuGWNpK/C+h9i1wU=)"
          }
        }
      },
      "status": 500
    }

```

**Server logs** :

```
[2020-08-24T23:27:27,288][WARN][r.suppressed] [node-5] path: /_snapshot/es5_aws_test_backup, params: {pretty=, repository=es5_aws_test_backup}
    org.elasticsearch.repositories.RepositoryException: [es5_aws_test_backup] cannot create blob store
            at org.elasticsearch.repositories.blobstore.BlobStoreRepository.blobStore(BlobStoreRepository.java:336) ~[elasticsearch-6.5.0.jar:6.5.0]
            at org.elasticsearch.repositories.s3.S3Repository.blobStore(S3Repository.java:258) ~[?:?]
            at org.elasticsearch.repositories.blobstore.BlobStoreRepository.startVerification(BlobStoreRepository.java:635) ~[elasticsearch-6.5.0.jar:6.5.0]
            at org.elasticsearch.repositories.RepositoriesService.lambda$verifyRepository$2(RepositoriesService.java:218) ~[elasticsearch-6.5.0.jar:6.5.0]
            at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) [elasticsearch-6.5.0.jar:6.5.0]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_231]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_231]
            at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]
    Caused by: java.lang.IllegalArgumentException: you do not have permissions to access the bucket [test-aws-backups]
            at org.elasticsearch.repositories.s3.S3BlobStore.lambda$new$0(S3BlobStore.java:79) ~[?:?]
            at org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:57) ~[?:?]
            at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_231]
            at org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:56) ~[?:?]
            at org.elasticsearch.repositories.s3.S3BlobStore.<init>(S3BlobStore.java:72) ~[?:?]
            at org.elasticsearch.repositories.s3.S3Repository.createBlobStore(S3Repository.java:251) ~[?:?]
            at org.elasticsearch.repositories.s3.S3Repository.createBlobStore(S3Repository.java:53) ~[?:?]
            at org.elasticsearch.repositories.blobstore.BlobStoreRepository.blobStore(BlobStoreRepository.java:332) ~[elasticsearch-6.5.0.jar:6.5.0]
            ... 7 more
    Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 0F6E6291D55D6322; S3 Extended Request ID: 6+O2pdgTQh+SYssTdRzisOlYAjHt3miNgA7SBL1eVm12ULRHA31Nw0TSoSqNtgcU6AzgvGdka58=)
            at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1658) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1322) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1072) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:745) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:719) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:701) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:669) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:651) ~[?:?]
            at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:515) ~[?:?]
            at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4443) ~[?:?]
            at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4390) ~[?:?]
            at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1354) ~[?:?]
            at org.elasticsearch.repositories.s3.S3BlobStore.lambda$new$0(S3BlobStore.java:74) ~[?:?]
            at org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:57) ~[?:?]
            at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_231]
            at org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:56) ~[?:?]
            at org.elasticsearch.repositories.s3.S3BlobStore.<init>(S3BlobStore.java:72) ~[?:?]
            at org.elasticsearch.repositories.s3.S3Repository.createBlobStore(S3Repository.java:251) ~[?:?]
            at org.elasticsearch.repositories.s3.S3Repository.createBlobStore(S3Repository.java:53) ~[?:?]
            at org.elasticsearch.repositories.blobstore.BlobStoreRepository.blobStore(BlobStoreRepository.java:332) ~[elasticsearch-6.5.0.jar:6.5.0]
            ... 7 more
```

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [August 25, 2020, 8:27am UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250/2 "2020-08-25T08:27:52Z")

</div>

Have you checked the auth permissions against the bucket?

---

<div class="post-metadata">

**Author:** ![dimitris\_sb](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@dimitris\_sb](https://discuss.elastic.co/u/dimitris_sb)\
**Post date:** [August 25, 2020, 8:35am UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250/3 "2020-08-25T08:35:43Z")

</div>

Thank you for your prompt reply!  
Can you please elaborate? How can I check that (except of trying to independently connect with another client)?

---

<div class="post-metadata">

**Author:** ![dimitris\_sb](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@dimitris\_sb](https://discuss.elastic.co/u/dimitris_sb)\
**Post date:** [August 25, 2020, 9:00am UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250/4 "2020-08-25T09:00:45Z")

</div>

From CyberDuck I can see:  
**ACL**  
Grantee Permission  
Admin FULL\_CONTROL  
Log Delivery READ\_ACP  
Log Delivery WRITE

Should I enforce a more specific policy?  
I understand that I should be able to check if the user which belongs to my account is able to access the bucket and list, put, delete objects.  
If I can do all these actions from a client why elasticsearch cannot do the same thing?

---

<div class="post-metadata">

**Author:** ![dimitris\_sb](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@dimitris\_sb](https://discuss.elastic.co/u/dimitris_sb)\
**Post date:** [August 25, 2020, 10:03am UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250/5 "2020-08-25T10:03:45Z")

</div>

I made a small python3 script using boto3 and I am able using the same credentials to access and write an object in the bucket.  
I cannot understand why elasticsearch using these credentials fails ... Is there a specific permission the user needs?

---

<div class="post-metadata">

**Author:** ![dimitris\_sb](https://avatars.discourse-cdn.com/v4/letter/d/eada6e/32.png) [@dimitris\_sb](https://discuss.elastic.co/u/dimitris_sb)\
**Post date:** [August 25, 2020, 3:44pm UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250/6 "2020-08-25T15:44:37Z")

</div>

Are this issue related?

> <https://github.com/elastic/elasticsearch/issues/35703>
>
> A user reports problems with the S3 repository plugin in 6.5.0:
> {
> "error": {
> "root\_cause": \[
> {
> "type": "repository\_exception",
> "reason": "\[s3\_test\] cannot...

  

> <https://github.com/elastic/elasticsearch/pull/35732>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2020, 3:44pm UTC](https://discuss.elastic.co/t/is-there-a-bug-in-elasticsearch-6-5-0-concerning-s3-snapshot-repositories/246250/7 "2020-09-22T15:44:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
