Is repository-s3 plugin fixed in 6.3?

Elastic Stack Elasticsearch
1

Running 6.2.4 here and repository-s3 plugin is not allowing me to create a repository despite the fact I have used the new elasticsearch-keystore binary to create both s3.client.default.access_key and s3.client.default.secret_key

{"error":{"root_cause":[{"type":"repository_exception","reason":"[my_s3_repository1] failed to create repository"}],"type":"repository_exception","reason":"[my_s3_repository1] failed to create repository","caused_by":{"type":"sdk_client_exception","reason":"Unable to load credentials from service endpoint","caused_by":{"type":"socket_timeout_exception","reason":"connect timed out"}}},"status":500}[root@cb5e3dd9d478

After hunting around, I'm thinking this might be fixed in the 6.3 release...

Can anyone confirm I'm reading this right?

2

Hello @windowsrefund_refund

I could not reproduce the problem you are describing. In general, it is advisable to paste the stack trace of the exception on the elasticsearch side to get a more accurate diagnosis.
However, I think I know what the problem is:
The keystore needs to be populated with

  bin/elasticsearch-keystore add s3.client.default.secret_key
  bin/elasticsearch-keystore add s3.client.default.access_key

before you start elasticsearch ./bin/elasticsearch.
If you have added the entries to the keystore after elasticsearch was started, you need to restart it.
Then, registering the repository should be a breeze:

curl -X PUT "localhost:9200/_snapshot/my_s3_repository" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "elasticsearch.test.bucket"
  }
}
'

NB: The GH issue you have pasted is not related to the matter at hand.

3

Hello and thanks for the response. I've set both s3.client.default.access_key and s3.client.default.secret_key using the elasticsearch-keystore binary on 1 of my nodes. This has been verified by running elasticsearch-keystore with the list option. However, upon restarting the elasticsearch service, I see this error in the log

at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
Caused by: java.lang.IllegalArgumentException: Setting [access_key] is insecure, but property [allow_insecure_settings] is not set

Is this allow_insecure_setting config needed? Docs certainly say nothing about it. Also, do I need to do this work on every node in the cluster or only 1? Docs say nothing about that as well.

4

OK so it turns out elasticsearch-keystore needs to be set on every node in the cluster. Why on earth this simple sentence is not present in the existing documentation is beyond me.

5

@windowsrefund_refund I am sorry for the frustration, I have opened https://github.com/elastic/elasticsearch/pull/31621

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.