I installed the cloud-aws plugin of elasticsearch version 2.3.5 which will be used to store the snapshot on AWS S3 compatible. During the install everything went nice and smoothly but when I try to create the snapshot I get the following error:
{
"error" : {
"root_cause" : [ {
"type" : "repository_exception",
"reason" : "[my_s3_repository] failed to create repository"
} ],
"type" : "repository_exception",
"reason" : "[my_s3_repository] failed to create repository",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Unknown [repository] type [s3]"
}
},
"status" : 500
}
I configured elasticsearch like below:
cloud:
aws:
protocol: https
access_key: ***********
secret_key: **********
endpoint: sth.com
proxy:
host: "http://192.168.0.5"
port: "3128"
s3:
protocol: http
signer: "AWS2SignerType"
proxy:
host: 192.168.0.5
port: 3128
You need to know that the endpoint requires Signature v2 enabled.
So in this case can you help me troubleshoot the error and confirm that the elasticsearch config is ok?
Into elasticsearch logs besides the error above I also see some java errors:
repository] type [s3]];
Caused by: RepositoryException[[my_s3_repository] failed to create repository]; nested: IllegalArgumentException[Unknown [repository] type [s3]];
at org.elasticsearch.repositories.RepositoriesService.createRepositoryHolder(RepositoriesService.java:411)
at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:368)
at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:55)
at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:110)
at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:468)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:772)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Unknown [repository] type [s3]
at org.elasticsearch.common.util.ExtensionPoint$SelectedType.bindType(ExtensionPoint.java:146)
at org.elasticsearch.repositories.RepositoryTypesRegistry.bindType(RepositoryTypesRegistry.java:49)
at org.elasticsearch.repositories.RepositoryModule.configure(RepositoryModule.java:58)
at org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:233)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:105)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:143)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:99)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(InjectorImpl.java:157)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(ModulesBuilder.java:55)
at org.elasticsearch.repositories.RepositoriesService.createRepositoryHolder(RepositoriesService.java:404)
Regarding the AWS2SignerType it's a parameter required by AWS, to have each http or https query formatted for signature version 2 or version 4 ( in our case the S3 compatible endpoint requires version 2).
But even if I don't set that parameter the error is the same, the plugin isn't able to recognize the "s3" type.
No, actually I have a cluster composed of 5 data nodes, 3 master nodes and 2 coordinating nodes. But I installed the plugin only on one data node and tried to create a snapshot just for it.
I install it on every node, but the error is the same.
Is there a specific configuration of this plugin for this version if I need to go through a proxy to reach the endpoint?
curl -XGET 10.0.1.87:9200/_cat/plugins?v
name component version type url
inlodmedat01 cloud-aws 2.3.5 j
And the logs:
[2017-11-03 13:11:55,025][WARN ][rest.suppressed ] path: /_snapshot/my_s3_repository, params: {pretty=, repository=my_s3_repository}
RemoteTransportException[[inlodmemas01][10.0.1.88:9300][cluster:admin/repository/put]]; nested: RepositoryException[[my_s3_repository] failed to create repository]; nested: IllegalArgumentException[Unknown [repository] type [s3]];
Caused by: RepositoryException[[my_s3_repository] failed to create repository]; nested: IllegalArgumentException[Unknown [repository] type [s3]];
at org.elasticsearch.repositories.RepositoriesService.createRepositoryHolder(RepositoriesService.java:411)
at org.elasticsearch.repositories.RepositoriesService.registerRepository(RepositoriesService.java:368)
at org.elasticsearch.repositories.RepositoriesService.access$100(RepositoriesService.java:55)
at org.elasticsearch.repositories.RepositoriesService$1.execute(RepositoriesService.java:110)
at org.elasticsearch.cluster.ClusterStateUpdateTask.execute(ClusterStateUpdateTask.java:45)
at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:468)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:772)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalArgumentException: Unknown [repository] type [s3]
at org.elasticsearch.common.util.ExtensionPoint$SelectedType.bindType(ExtensionPoint.java:146)
at org.elasticsearch.repositories.RepositoryTypesRegistry.bindType(RepositoryTypesRegistry.java:49)
at org.elasticsearch.repositories.RepositoryModule.configure(RepositoryModule.java:58)
at org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:60)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:233)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:105)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:143)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:99)
at org.elasticsearch.common.inject.InjectorImpl.createChildInjector(InjectorImpl.java:157)
at org.elasticsearch.common.inject.ModulesBuilder.createChildInjector(ModulesBuilder.java:55)
at org.elasticsearch.repositories.RepositoriesService.createRepositoryHolder(RepositoriesService.java:404)
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.