ElasticSearch Snapshot to S3

Hello Community,

Not able to register new repository snapshot to S3.

Request

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository?verify=false&pretty' -d'
{
"type": "s3",
"settings": {
"bucket": "***",
"region": "ap-south-1",
"access_key": "",
"secret_key": "
"
}
}' -H "Content-Type:application/json"

Logs
{
"error" : {
"root_cause" : [
{
"type" : "repository_exception",
"reason" : "[s3_repository] failed to create repository"
}
],
"type" : "repository_exception",
"reason" : "[s3_repository] failed to create repository",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Setting [access_key] is insecure, but property [allow_insecure_settings] is not set"
}
},
"status" : 500
}

You can't use anymore access_key / secret_key in the repository settings because anyone can read your credentials then. Which makes that insecure.

If you still want to do it, set allow_insecure_settings: true in your elasticsearch.yml file.

Better, use secured settings: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/secure-settings.html

As explained in https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-client.html

Hi Dadoonet,

I tried adding "allow_insecure_settings: true in my elasticsearch.yml file", but after elasticsearch restart I am getting following errors in the logs.

Exception java.lang.IllegalArgumentException: unknown setting [allow_insecure_settings] please check that any required plugins are installed, or check the breaking changes documentation for removed settings

And when I was setting access.key and secret.key (according to the blog provided ) then having following problem --->

Request

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository' -d'
> {
>   "type": "s3",
>   "settings": {
>     "bucket": "*****",
>     "region": "ap-south-1"
>   } 
> }' -H "Content-Type:application/json" 

Response

{
  "acknowledged" : true
}

Request
GET /_snapshot/_all

Response
"Empty response"

hence not able to PUT any repository in the given bucket.

Can you share the exact steps you did to add the keys and the repository?
Please take that from your history. Don't copy/paste the documentation but what you did exactly.

Also, share elasticsearch logs.

In case anyone stumbles this thread,

echo "-Des.allow_insecure_settings=true" >> /usr/share/elasticsearch/config/jvm.options should fix the issue.