ElasticSearch Snapshot to S3

[pratsy (Pratyush Agarwal)]

Hello Community,

Not able to register new repository snapshot to S3.

Request

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository?verify=false&pretty' -d'
{
"type": "s3",
"settings": {
"bucket": "***",
"region": "ap-south-1",
"access_key": "",
"secret_key": ""
}
}' -H "Content-Type:application/json"

Logs
{
"error" : {
"root_cause" : [
{
"type" : "repository_exception",
"reason" : "[s3_repository] failed to create repository"
}
],
"type" : "repository_exception",
"reason" : "[s3_repository] failed to create repository",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Setting [access_key] is insecure, but property [allow_insecure_settings] is not set"
}
},
"status" : 500
}

[dadoonet (David Pilato)]

You can't use anymore access_key / secret_key in the repository settings because anyone can read your credentials then. Which makes that insecure.

If you still want to do it, set allow_insecure_settings: true in your elasticsearch.yml file.

Better, use secured settings: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/secure-settings.html

As explained in https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-client.html

[pratsy (Pratyush Agarwal)]

Hi Dadoonet,

I tried adding "allow_insecure_settings: true in my elasticsearch.yml file", but after elasticsearch restart I am getting following errors in the logs.

Exception java.lang.IllegalArgumentException: unknown setting [allow_insecure_settings] please check that any required plugins are installed, or check the breaking changes documentation for removed settings

[pratsy (Pratyush Agarwal)]

And when I was setting access.key and secret.key (according to the blog provided ) then having following problem --->

Request

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository' -d'
> {
>   "type": "s3",
>   "settings": {
>     "bucket": "*****",
>     "region": "ap-south-1"
>   } 
> }' -H "Content-Type:application/json" 

Response

{
  "acknowledged" : true
}

Request
GET /_snapshot/_all

Response
"Empty response"

hence not able to PUT any repository in the given bucket.

[dadoonet (David Pilato)]

Can you share the exact steps you did to add the keys and the repository?
Please take that from your history. Don't copy/paste the documentation but what you did exactly.

Also, share elasticsearch logs.