S3-repository deprecated settings warning

[yee]

Hi,
I created a snapshot in ElasticSearch 5.4.3, according to https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-repository.html , my cmd as below:

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository' -d '{
    "type": "s3",
    "settings": {
        "access_key": "***",
        "secret_key": "***",
        "bucket": "es-snapshots",
        "region": "cn-north-1",
        "base_path": "Elasticsearch",
        "max_retries": 3
    }
}'

But ElasticSearch writes lots of WARNING logs out:

[2017-07-08T15:28:52,770][WARN ][org.elasticsearch.deprecation.common.settings.Setting] [region] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.
[2017-07-08T15:28:52,771][WARN ][org.elasticsearch.deprecation.repositories.s3.InternalAwsS3Service] Specifying region for an s3 repository is deprecated. Use endpoint to specify the region endpoint if the default behavior is not sufficient.
[2017-07-08T15:28:52,773][WARN ][org.elasticsearch.deprecation.repositories.s3.InternalAwsS3Service] Using s3 access/secret key from repository settings. Instead store these in named clients and the elasticsearch keystore for secure settings.
[2017-07-08T15:28:52,787][WARN ][org.elasticsearch.deprecation.common.settings.Setting] [access_key] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.
[2017-07-08T15:28:52,787][WARN ][org.elasticsearch.deprecation.common.settings.Setting] [secret_key] setting was deprecated in Elasticsearch and will be removed in a future release! See the breaking changes documentation for the next major version.

I wonder what's wrong with my operations, or the doc needs to be updated?

[dadoonet (David Pilato)]

Doc needs an update.

You should start using endpoint instead.

[dadoonet (David Pilato)]

You should also start using the keystore to store credentials.

Secure Settings | Elasticsearch Reference [5.5] | Elastic

Then add your secure settings as:

s3.client.default.access_key
s3.client.default.secret_key

See

Getting AWS credentials from keystore for discovery-ec2 plugin?

I see. Sadly the documentation has not been updated yet. It needs to be: discovery.ec2.access_key discovery.ec2.secret_key cloud.aws.region should not be used anymore but discovery.ec2.endpoint needs to be set in elasticsearch.yml file. See https://github.com/elastic/elasticsearch/blob/5.4/plugins/discovery-ec2/src/main/java/org/elasticsearch/discovery/ec2/AwsEc2Service.java#L181-L188 I hope this helps. cc @rjernst in case I'm saying something wrong

And

Getting AWS credentials from keystore for discovery-ec2 plugin?

The different keystore settings did affect the result, now able to create bucket. Here is keystore list output: /mnt/elasticsearch# bin/elasticsearch-keystore list discovery.ec2.access_key discovery.ec2.secret_key s3.client.default.access_key s3.client.default.secret_key Here is result of running shell script to create the repository bucket: /mnt/elasticsearch# bin/s3-repo-create.sh es-lineartv_test {"acknowledged":true}

[dadoonet (David Pilato)]

I opened the following issue:

Issue: Update S3/EC2 Documentation