Cloud-aws plugin with ceph bug?

Elastic Stack Elasticsearch
1

Hello,

I want to be able to create backup and restore using backup from ceph. I am using elasticsearch 2.0.

To create backup I use these steps:
create repository:
curl -XPUT 'http://localhost:9200/_snapshot/ceph_repository' -d '{"type": "s3", "settings": { "bucket": "mybucket", "endpoint": "myendpoint.com", "protocol":"http", "access_key": "my_access_key", "secret_key": "my_secret_key", "base_path": "my/base/path" }}'

to create backup
curl -XPUT 'http://localhost:9200/_snapshot/ceph_repository/snap1?wait_for_completion=true'

which outputs json containing "status":"SUCCESS" and files are created in ceph bucket.

To restore I use these steps:
Delete elastic search data directory and restart elastic search (after restarting cluster health gives "status" : "green").

Create repository using previously used command.
Restore using ceph snapshot:
curl -XPOST 'http://localhost:9200/_snapshot/ceph_repository/snap1/_restore'

and this gives an error:
{"error":{"root_cause":[{"type":"amazon_s3_exception","reason":"null (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: null)"}],"type":"amazon_s3_exception","reason":"null (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: null)"},"status":500}

Trying to debug:
curl -XGET 'http://localhost:9200/_snapshot/_all'
gives:
{"ceph_repository":{"type":"s3","settings":{"bucket":"mybucket","base_path":"my/base/path","endpoint":"myendpoint.com","protocol":"http"}}}

curl -XGET 'http://localhost:9200/_snapshot/_status'
gives:
{"snapshots":[]}

curl -XGET 'http://localhost:9200/_snsitory/snap1/_status'
gives:
{"error":{"root_cause":[{"type":"amazon_s3_exception","reason":"null (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: null)"}],"type":"amazon_s3_exception","reason":"null (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: null)"},"status":500}

I ommited elasticsearch.yml config since there is no configuration related to cloud-aws plugin.

2

I think it has been fixed in 2.1.0. with this PR: https://github.com/elastic/elasticsearch/pull/13360

And this doc: https://www.elastic.co/guide/en/elasticsearch/plugins/current/cloud-aws-usage.html#cloud-aws-usage-signer

3

I will try to repeat those actions with latest (2.2) version of elasticsearch. I am curious on how my settings should look like. Currently I do not use any of the cloud-aws plugin settings in my elasticsearch.yml config because of these reasons:
cloud.aws - is not used because I do not use any of amazon web services.
repositories.s3 - because I don't really know what is the purpose of it (since you still need to create repositories manually).

4

Even after changing version to 2.2 after snapshot is uploaded to ceph command:

curl -XGET 'http://localhost:9200/_snapshot/ceph_repository/snap1/_status'

gives:
{
"error": {
"reason": "null (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: null)",
"root_cause": [
{
"reason": "null (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: null)",
"type": "amazon_s3_exception"
}
],
"type": "amazon_s3_exception"
},
"status": 500
}

5

I was missing correct signer type:
"signer": "S3SignerType"
Thanks for help!