S3 repository plugin: Is it possible to restore a snapshot from a non-read-only repository using a read-only access_key?

Hello all,

I am trying to restore a snapshot from an s3 bucket created with Elasticsearch 2.2.1, using Elasticsearch 5.0.2. The bucket is not read-only, but the access_key for AWS only has read-only access.

When I try to restore a snapshot using:

POST http://localhost:9200/_snapshot/repo_name/snapshot_name/_restore
{
  "index_settings": {
    "index.number_of_replicas": 0
  }
}

It produces the following error:

{
  "error": {
    "root_cause": [
      {
        "type": "repository_exception",
        "reason": "[repo_name] failed to update the repository index blob with indices data on startup"
      }
    ],
    "type": "repository_exception",
    "reason": "[repo_name] failed to update the repository index blob with indices data on startup",
    "caused_by": {
      "type": "i_o_exception",
      "reason": "Unable to upload **** (anonymized this),
      "caused_by": {
        "type": "amazon_s3_exception",
        "reason": "Access Denied (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 1AE8CE8E63CB4BB8)"
      }
    }
  },
  "status": 500
}

If I search for that particular error message in the source code, it seems like this exception is thrown when the repository is not read-only and it tries to update some information in that repository, which of course is not allowed if you have an AWS access_key with read-only access.

So my question is: Is there a way to restore that snapshot using a read-only AWS access_key? Or do I have to have an AWS access_key with write access?

Try to create the repo with readonly instead of read_only. There is a bug in the docs.

The repo is not read-only, and is supposed to be that. It is also hosted by another company, so I cannot change it even if I wanted to.

My question still remains: can you do a read-only snapshot restore on a non-read-only repo?

How do you create the repository in your elasticsearch cluster?

I create the repo in my cluster like this (using fake names in this example):

PUT http://localhost:9200/_snapshot/repo_name?verify=false
{
  "type": "s3",
  "settings": {
    "bucket": "bucket_name",
    "access_key": "*******",
    "secret_key": "*******",
    "base_path": "some/path",
    "compress": "true",
  }
}

I am aware that I use verify=false. I need to, because verification will fail for the same reason, which is that my AWS access_key is read-only. In version 2.X however, the snapshot restore would still work. In version 5.0.2, I can't get it to work, that's why I am asking if there is something I am missing, or if I am running into a (new or old) limitation of the snapshot restore.

Can you try:

PUT http://localhost:9200/_snapshot/repo_name?verify=false
{
  "type": "s3",
  "settings": {
    "bucket": "bucket_name",
    "access_key": "*******",
    "secret_key": "*******",
    "base_path": "some/path",
    "compress": "true",
    "readonly": true
  }
}

I was just doing that, out of curiosity, and that is indeed what I was looking for. I just was misunderstanding how a repository works in elasticsearch. I thought it was just a pointer to the bucket in S3 (which is hosted by the other company), but I am actually creating a bit more than that in my own cluster, or so it seems.

Thanks David, for your help!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.