# Can't create s3 repository for backup in AWS Beijing region

**URL:** <https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584>\
**Category:** Elasticsearch\
**Created:** [July 25, 2018, 1:27pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584 "2018-07-25T13:27:31Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![k-troska](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/k-troska/32/67374_2.png) [@k-troska](https://discuss.elastic.co/u/k-troska)\
**Post date:** [July 25, 2018, 1:27pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584/1 "2018-07-25T13:27:31Z")

</div>

**Elasticsearch version** (`bin/elasticsearch --version`):  
Version: 6.3.2, Build: oss/deb/053779d/2018-07-20T05:20:23.451332Z, JVM: 1.8.0\_171

**Plugins installed** :  
ingest-geoip  
ingest-user-agent  
repository-s3

**JVM version** (`java -version`):  
openjdk version "1.8.0\_171"  
OpenJDK Runtime Environment (build 1.8.0\_171-8u171-b11-1~bpo8+1-b11)  
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)

**OS version** (`uname -a` if on a Unix-like system):  
Linux [name redacted] 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u3 (2017-08-15) x86\_64 GNU/Linux

**Description of the problem including expected versus actual behavior** :  
Can't add repository-s3 in cn-north-1 region (Beijing), with following error:  
**Request**  
`curl -s -X PUT "http://[hostname]:9200/_snapshot/s3_repository?pretty" -d @repo.json -H "Content-type:application/json"`  
**repo.json**  
`{ "type": "s3", "settings": { "bucket": "[name redacted]" } }`  
**Output**

```auto
{
  "error" : {
    "root_cause" : [
      {
        "type" : "repository_exception",
        "reason" : "[s3_repository] failed to create repository"
      }
    ],
    "type" : "repository_exception",
    "reason" : "[s3_repository] failed to create repository",
    "caused_by" : {
      "type" : "amazon_s3_exception",
      "reason" : "Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: C171506AC1F99990; S3 Extended Request ID: K8qtQddMD7JX+Y/B58oz04rE/HlPeHlxQyuK0e3HLSHk0e3HhzWSL2sBC+OJ+tj2BXH4EXfFsP4=)"
    }
  },
  "status" : 500
}

```

**Steps to reproduce** :

Please include a _minimal_ but _complete_ recreation of the problem, including  
(e.g.) index creation, mappings, settings, query etc. The easier you make for  
us to reproduce it, the more likely that somebody will take the time to look at it.

1. install elasticsearch-oss
2. install repository-s3 plugin
3. try to add repository

**Provide logs (if relevant)**:

Stack trace:

```auto
[2018-07-25T19:39:02,255][WARN][r.suppressed] path: /_snapshot/s3_repository, params: {pretty=, repository=s3_repository}
org.elasticsearch.repositories.RepositoryException: [s3_repository] failed to create repository
	at org.elasticsearch.repositories.RepositoriesService.createRepository(RepositoriesService.java:388) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:356) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:55) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:108) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:630) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.cluster.service.MasterService.calculateTaskOutputs(MasterService.java:267) ~[elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.cluster.service.MasterService.runTasks(MasterService.java:197) [elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.cluster.service.MasterService$Batcher.run(MasterService.java:132) [elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) [elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) [elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:626) [elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:244) [elasticsearch-6.3.2.jar:6.3.2]
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:207) [elasticsearch-6.3.2.jar:6.3.2]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: C76856AFD42A7C18; S3 Extended Request ID: Dwj3TJ9dSlw6KYB4Xq9SZEbpntDIx+4upQcq2krrl2dktf2flSJV0u2Kbpm4QGy0XfxhKOCj9tM=)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649) ~[?:?]
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513) ~[?:?]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4247) ~[?:?]
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4194) ~[?:?]
	at com.amazonaws.services.s3.AmazonS3Client.headBucket(AmazonS3Client.java:1326) ~[?:?]
	at com.amazonaws.services.s3.AmazonS3Client.doesBucketExist(AmazonS3Client.java:1266) ~[?:?]
	at org.elasticsearch.repositories.s3.S3BlobStore.lambda$new$0(S3BlobStore.java:72) ~[?:?]
	at org.elasticsearch.repositories.s3.SocketAccess.lambda$doPrivilegedVoid$0(SocketAccess.java:57) ~[?:?]
	at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_171]
	at org.elasticsearch.repositories.s3.SocketAccess.doPrivilegedVoid(SocketAccess.java:56) ~[?:?]
	at org.elasticsearch.repositories.s3.S3BlobStore.<init>(S3BlobStore.java:71) ~[?:?]
	at org.elasticsearch.repositories.s3.S3Repository.<init>(S3Repository.java:183) ~[?:?]
	at org.elasticsearch.repositories.s3.S3RepositoryPlugin.lambda$getRepositories$1(S3RepositoryPlugin.java:76) ~[?:?]
	at org.elasticsearch.repositories.RepositoriesService.createRepository(RepositoriesService.java:383) ~[elasticsearch-6.3.2.jar:6.3.2]
	... 16 more

```

---

<div class="post-metadata">

**Author:** ![k-troska](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/k-troska/32/67374_2.png) [@k-troska](https://discuss.elastic.co/u/k-troska)\
**Post date:** [July 25, 2018, 1:30pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584/2 "2018-07-25T13:30:14Z")

</div>

Generated with help of: [https://github.com/elastic/ansible-elasticsearch](https://github.com/elastic/ansible-elasticsearch)  
cat /etc/elasticsearch/es01/elasticsearch.yml

```auto

bootstrap.memory_lock: true
cluster.name: APAC-logs
cluster.routing.allocation.awareness.attributes: zone
cluster.routing.allocation.awareness.force.zone.values: cn-north-1a,cn-north-1b
discovery.zen.ping.unicast.hosts: es01.[redacted],es02.[redacted]
network.host: 10.16.40.182
node.attr.zone: cn-north-1a
node.data: true
node.master: true
node.name: es01

#################################### Paths ####################################

# Path to directory containing configuration (this file and logging.yml):

path.data: /var/lib/elasticsearch/es01.elk.[redacted]-es01

path.logs: /var/log/elasticsearch/es01.elk.[redacted]-es01

action.auto_create_index: true

```

For permission I am using Iam policy/role:

```auto
{
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucketVersions"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws-cn:s3:::[redacted]"
            ]
        },
        {
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws-cn:s3:::[redacted]/*"
            ]
        }
    ],
    "Version": "2012-10-17"
}

```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 25, 2018, 1:55pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584/3 "2018-07-25T13:55:20Z")

</div>

I already linked to [https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-client.html](https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-client.html) in the github issue you opened.

Set `endpoint`.

---

<div class="post-metadata">

**Author:** ![k-troska](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/k-troska/32/67374_2.png) [@k-troska](https://discuss.elastic.co/u/k-troska)\
**Post date:** [July 25, 2018, 2:50pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584/4 "2018-07-25T14:50:09Z")

</div>

Thank you it helped, seams to work after I added  
`s3.client.default.endpoint: s3.cn-north-1.amazonaws.com.cn`  
to elasticsearch.yml on both nodes and restarted them.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 25, 2018, 5:03pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584/5 "2018-07-25T17:03:55Z")

</div>

I'm curious about the bucket name you used then?  
Any chance you can DM it to me if you don't want to share it publicly?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [July 31, 2018, 12:57pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584/6 "2018-07-31T12:57:20Z")

</div>

For the record I opened this PR to fix the documentation:

> <https://github.com/elastic/elasticsearch/pull/32497>

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 28, 2018, 12:57pm UTC](https://discuss.elastic.co/t/cant-create-s3-repository-for-backup-in-aws-beijing-region/141584/7 "2018-08-28T12:57:25Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.