# S3 Repository Config **URL:** https://discuss.elastic.co/t/s3-repository-config/132345 **Category:** Elasticsearch **Created:** [May 17, 2018, 1:38pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345 "2018-05-17T13:38:49Z") **Posts on this page:** 20 **Page:** 1
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 17, 2018, 1:38pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/1 "2018-05-17T13:38:49Z")
Hi, I'm getting an error provisioning an s3 repo on elasticsearch 5.6.3 via repository-s3. Can you assist? config json: ``` { "type": "s3", "settings": { "endpoint": "s3-server", "bucket": "elasticsearch", "client": "lab" } } ``` The error I get is: ``` { "error": { "root_cause": [ { "type": "amazon_s3_exception", "reason": "amazon_s3_exception: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: tx00000000000000000e8f7-005afd82b2-c73109-default)" } ], "type": "repository_exception", "reason": "[s3-snapshots] failed to create repository", "caused_by": { "type": "amazon_s3_exception", "reason": "amazon_s3_exception: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: tx00000000000000000e8f7-005afd82b2-c73109-default)" } }, "status": 500 } ``` Note that the S3 server is ceph. In elasticsearch 2.x i would specify a `signertype` of `S3SignerType`. How do I do this for repository-s3? Regards, David ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [May 17, 2018, 2:12pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/2 "2018-05-17T14:12:58Z")
I don't think this plugin is compatible with CEPH. ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 17, 2018, 2:15pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/3 "2018-05-17T14:15:29Z")
The cloud-aws plugin was compatible. I'm surprised there isn't feature parity. ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [May 17, 2018, 2:31pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/4 "2018-05-17T14:31:19Z")
It was untested so compatible by luck I'd say. Best thing to do IMO is to fork the S3 plugin and add whatever is needed to make it work with old S3 API. ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 17, 2018, 2:42pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/5 "2018-05-17T14:42:55Z")
Can't backward compatibility be added as a feature? ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [May 17, 2018, 2:59pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/6 "2018-05-17T14:59:36Z")
We don't want to add a feature we are not testing. May be it will come in the future or someone from the community will build it. For reference see: > ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 18, 2018, 6:48am UTC](https://discuss.elastic.co/t/s3-repository-config/132345/7 "2018-05-18T06:48:08Z")
Ok. Where can I find guidance on how to get started with this? I've never done it before... ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 18, 2018, 8:08am UTC](https://discuss.elastic.co/t/s3-repository-config/132345/8 "2018-05-18T08:08:04Z")
So, seeing as I'm running 5.6.3, signer settings won't be an issue for me (yet)? ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [May 18, 2018, 8:23am UTC](https://discuss.elastic.co/t/s3-repository-config/132345/9 "2018-05-18T08:23:55Z")
That should work indeed. You should see only some deprecation messages I guess. ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 18, 2018, 8:24am UTC](https://discuss.elastic.co/t/s3-repository-config/132345/10 "2018-05-18T08:24:33Z")
I've pressed on, am getting different errors now. Config looks like this: ``` cloud.aws.s3.endpoint: s3-server cloud.aws.s3.signer: S3SignerType ``` Repo config json: ``` { "type": "s3", "settings": { "access_key": "access-key-string", "secret_key": "secret-key-string", "endpoint": "s3-server", "bucket": "elasticsearch", "client": "lab", "base_path": "snapshots/elasticsearch_lab", "protocol": "http", "compress": "false", "read_timeout": "15s", "storage-class": "default", "chunk_size": "1gb" } } ``` And the logged exception: ``` Caused by: java.lang.IllegalArgumentException: Can not access public com.amazonaws.partitions.model.Partitions(java.lang.String,java.util.List) (from class com.amazonaws.partitions.model.Partitions; failed to set access: access denied ("java.lang.reflect.ReflectPermission" "suppressAccessChecks") at com.fasterxml.jackson.databind.util.ClassUtil.checkAndFixAccess(ClassUtil.java:505) ~[?:?] at com.fasterxml.jackson.databind.deser.impl.CreatorCollector._fixAccess(CreatorCollector.java:271) ~[?:?] at com.fasterxml.jackson.databind.deser.impl.CreatorCollector.verifyNonDup(CreatorCollector.java:318) ~[?:?] at com.fasterxml.jackson.databind.deser.impl.CreatorCollector.addPropertyCreator(CreatorCollector.java:183) ~[?:?] at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._addDeserializerConstructors(BasicDeserializerFactory.java:497) ~[?:?] at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory._constructDefaultValueInstantiator(BasicDeserializerFactory.java:325) ~[?:?] at com.fasterxml.jackson.databind.deser.BasicDeserializerFactory.findValueInstantiator(BasicDeserializerFactory.java:258) ~[?:?] at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.buildBeanDeserializer(BeanDeserializerFactory.java:216) ~[?:?] at com.fasterxml.jackson.databind.deser.BeanDeserializerFactory.createBeanDeserializer(BeanDeserializerFactory.java:143) ~[?:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer2(DeserializerCache.java:409) ~[?:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createDeserializer(DeserializerCache.java:358) ~[?:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:265) ~[?:?] at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:245) ~[?:?] at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:143) ~[?:?] at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:439) ~[?:?] at com.fasterxml.jackson.databind.ObjectMapper._findRootDeserializer(ObjectMapper.java:3666) ~[?:?] at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3558) ~[?:?] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2648) ~[?:?] at com.amazonaws.partitions.PartitionsLoader.loadPartitionFromStream(PartitionsLoader.java:96) ~[?:?] at com.amazonaws.partitions.PartitionsLoader.build(PartitionsLoader.java:88) ~[?:?] at com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30) ~[?:?] at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:66) ~[?:?] at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:54) ~[?:?] at com.amazonaws.regions.RegionUtils.getRegionByEndpoint(RegionUtils.java:123) ~[?:?] at com.amazonaws.services.s3.AmazonS3Client.createBucket(AmazonS3Client.java:725) ~[?:?] at org.elasticsearch.repositories.s3.S3BlobStore.(S3BlobStore.java:86) ~[?:?] at org.elasticsearch.repositories.s3.S3Repository.(S3Repository.java:308) ~[?:?] at org.elasticsearch.repositories.s3.S3RepositoryPlugin.lambda$getRepositories$0(S3RepositoryPlugin.java:82) ~[?:?] at org.elasticsearch.repositories.RepositoriesService.createRepository(RepositoriesService.java:383) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:356) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:56) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:109) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cluster.service.ClusterService.executeTasks(ClusterService.java:634) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cluster.service.ClusterService.calculateTaskOutputs(ClusterService.java:612) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cluster.service.ClusterService.runTasks(ClusterService.java:571) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cluster.service.ClusterService$ClusterServiceTaskBatcher.run(ClusterService.java:263) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cluster.service.TaskBatcher.runIfNotProcessed(TaskBatcher.java:150) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.cluster.service.TaskBatcher$BatchedTask.run(TaskBatcher.java:188) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:569) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:247) ~[elasticsearch-5.6.3.jar:5.6.3] at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:210) ~[elasticsearch-5.6.3.jar:5.6.3] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_171] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_171] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_171] ``` ---
### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet) #### Post date: [May 18, 2018, 8:45am UTC](https://discuss.elastic.co/t/s3-repository-config/132345/11 "2018-05-18T08:45:03Z")
Sounds like a Security issue. It needs probably some other permissions than the ones the plugin is using for S3 at it uses probably different classes. I think you need to modify the `plugin-security.policy` file. @tlrx might know more about this. ---
### Author: ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy) #### Post date: [May 18, 2018, 9:11am UTC](https://discuss.elastic.co/t/s3-repository-config/132345/12 "2018-05-18T09:11:16Z")
@dawiro: Which JVM are you using exactly? (it's printed out in the elasticsearch logs at startup time) ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 18, 2018, 9:38am UTC](https://discuss.elastic.co/t/s3-repository-config/132345/13 "2018-05-18T09:38:15Z")
JVM info: ``` $ java -version openjdk version "1.8.0_171" OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-0ubuntu0.16.04.1-b11) OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode) ``` ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 18, 2018, 12:07pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/14 "2018-05-18T12:07:35Z")
@tanguy @dadoonet see jvm info above 🙂 ---
### Author: ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy) #### Post date: [May 18, 2018, 12:12pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/15 "2018-05-18T12:12:50Z")
Thanks. Does the bucket already exists on Ceph? ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 18, 2018, 12:13pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/16 "2018-05-18T12:13:54Z")
No it doesn't @tanguy . ---
### Author: ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy) #### Post date: [May 18, 2018, 12:58pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/17 "2018-05-18T12:58:18Z")
Can you create it on Ceph and try to register the repository again? Note that on more recent versions of Elasticsearch, the bucket is not created automatically anymore. ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 18, 2018, 2:23pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/18 "2018-05-18T14:23:34Z")
@tanguy I'll try that. Why has that been changed? It doesn't make sense. ---
### Author: ![tanguy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tanguy/32/6030_2.png) [@tanguy](https://discuss.elastic.co/u/tanguy) #### Post date: [May 18, 2018, 2:40pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/19 "2018-05-18T14:40:58Z")
> [@dawiro](#): > > Why has that been changed? It doesn't make sense. What do you mean by "that"? ---
### Author: ![dawiro](https://avatars.discourse-cdn.com/v4/letter/d/71e660/32.png) [@dawiro](https://discuss.elastic.co/u/dawiro) #### Post date: [May 25, 2018, 12:06pm UTC](https://discuss.elastic.co/t/s3-repository-config/132345/20 "2018-05-25T12:06:17Z")
I mean that it makes sense to me that either: - repo creation not depend on the bucket existing - the repo creates the bucket when it is created, if the bucket does not exist [Next page](https://discuss.elastic.co/t/s3-repository-config/132345.md?page=2)