We are upgrading to ES 5.4.x after having used 1.7.3 and 2.3.3 in production. Having learned that including AWS credentials in config/elasticsearch.yml has been discouraged, I setup a keystore to hold those settings. But it seems the discovery-ec2 plugin is not using that resource to obtain credentials. Am I missing a setting somewhere to allow that behavior?
It should work OOTB. What exactly did you configure in the keystore? Can you list the settings you have?
I added these settings: (command and output)
./elasticsearch-keystore list
cloud.aws.access_key
cloud.aws.region
cloud.aws.secret_key
Using chef to provision the host, executed commands as follows:
#{es_home_dir}/bin/elasticsearch-keystore create
/bin/echo #{es_aws_access_key} | #{es_home_dir}/bin/elasticsearch-keystore add --stdin cloud.aws.access_key
/bin/echo #{es_aws_secret_access_key} | #{es_home_dir}/bin/elasticsearch-keystore add --stdin cloud.aws.secret_key
/bin/echo 'us-east-1' | #{es_home_dir}/bin/elasticsearch-keystore add --stdin cloud.aws.region
When I issue the list command, should I also see the values with the keys?
I see. Sadly the documentation has not been updated yet. It needs to be:
discovery.ec2.access_keydiscovery.ec2.secret_key
cloud.aws.region should not be used anymore but discovery.ec2.endpoint needs to be set in elasticsearch.yml file.
I hope this helps.
cc @rjernst in case I'm saying something wrong 
Unfortunately, changing keystore names did not solve the issue. We have solved the problem by entering the updated key names in elasticsearch.yml.
@Jaff What do you mean by We have solved the problem by entering the updated key names in elasticsearch.yml? David is correct that the keystore settings should be discovery.ec2.access_key and discovery.ec2.secret_key. Any of the old settings will not work there (unfortunately we do not yet have validation of settings in the keystore, so you will not get an error if you add settings that should not be there). But setting eg discovery.ec2.access_key in elasticsearch.yml should not work. This setting should only be allowed in the keystore.
We are OK with discovery-ec2 plugin, however repository-s3 authentication is failing. I have set keystore values for cloud.aws.s3.access_key and cloud.aws.s3.secret_key and still getting failure.
Ok, repository-s3 has different settings. This is due to that plugin allow for multiple credential sets. There you need to set s3.client.default.access_key and s3.client.default.secret_key.
1 Like
The different keystore settings did affect the result, now able to create bucket. Here is keystore list output:
/mnt/elasticsearch# bin/elasticsearch-keystore list
discovery.ec2.access_key
discovery.ec2.secret_key
s3.client.default.access_key
s3.client.default.secret_key
Here is result of running shell script to create the repository bucket:
/mnt/elasticsearch# bin/s3-repo-create.sh es-lineartv_test
{"acknowledged":true}
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.