Create snapshot in s3 - issue to create snapshot in region eu-west-2

Elastic Stack Elasticsearch
1

Hi guys,

I'm trying to create es snapshots in s3, region: eu-west-2 (London) and facing failure.
Please see below more info:

es version:

curl -XGET 'localhost:9200'
{
  "name" : "node-name",
  "cluster_name" : "es-cluster",
  "cluster_uuid" : "IYV8G-gSRmqkQLsSw2OXwQ",
  "version" : {
    "number" : "2.4.6",
    "build_hash" : "5376dca9f70f3abef96a77f4bb22720ace8240fd",
    "build_timestamp" : "2017-07-18T12:17:44Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.4"
  },
  "tagline" : "You Know, for Search"

when trying to create snapshot in other region (eu-west-1 for example), it looks good:

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository?verify=false&pretty' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "bucket-snapshot",
    "region": "eu-west-1"
}
}'

response is:

{
  "acknowledged" : true
}

but, when trying the same but in region eu-west-2 - it failed:

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository?verify=false&pretty' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "bucket-snapshot",
    "region": "eu-west-2"
  }
}'

response is:

{
  "error" : {
    "root_cause" : [ {
      "type" : "repository_exception",
      "reason" : "[s3_repository] failed to create repository"
    } ],
    "type" : "repository_exception",
    "reason" : "[s3_repository] failed to create repository",
    "caused_by" : {
      "type" : "creation_exception",
      "reason" : "Guice creation errors:\n\n1) Error injecting constructor, java.lang.IllegalArgumentException: No automatic endpoint could be derived from region [eu-west-2]\n  at org.elasticsearch.repositories.s3.S3Repository.<init>(Unknown Source)\n  while locating org.elasticsearch.repositories.s3.S3Repository\n  while locating org.elasticsearch.repositories.Repository\n\n1 error",
      "caused_by" : {
        "type" : "illegal_argument_exception",
        "reason" : "No automatic endpoint could be derived from region [eu-west-2]"
      }
    }
  },
  "status" : 500
}

can you please assist with issue ? what is wrong here ?

Thank you,
Ron

2

Elasticsearch 2.4 uses AWS SDK version 1.10.69, released no later than April 2016, whereas the AWS eu-west-2 region was launched in December 2016, so needs a newer SDK.

3

thank you !

4

That said, it's probably possible to use the endpoint instead of region.
This has been implemented in the plugin to make that possible. See https://www.elastic.co/guide/en/elasticsearch/plugins/2.4/cloud-aws-repository.html

So probably this should work although the API version might not be compatible. I mean that you need to test it :wink: :

curl -XPUT 'http://localhost:9200/_snapshot/s3_repository?verify=false&pretty' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "bucket-snapshot",
    "endpoint": "s3.eu-west-2.amazonaws.com"
  }
}'
5

I would be very interested to hear if this does work. I did not suggest setting the endpoint because AIUI the signature algorithm apparently requires the region name and I expected that setting the endpoint explicitly would leave the region blank, but I have neither tried it nor traced through the code to verify how it sets the region in this case. If it does work, please let us know!

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.