Repository-s3 is not compatible with AWS S3 on OUTPOSTS

Elastic Stack Elasticsearch
1

Hello!

I am running Elasticsearch cluster on AWS EC2s in version 7.9.1 on AWS Outposts. I wanted to register snapshot repository on s3 bucket which is in this same Outposts, and I was not able to do it.

S3 on outposts requires AccessPoint to connect to. Actually it is possible to register snapshot repository on s3 through AccessPoint either by using AP ARN or alias but for Outpost it is not possible in any way:

{
  "type": "s3",
  "settings": {
  "bucket": "es-snapshots-*******--op-s3",
  "region" : "us-west-2",
  "endpoint" : "es-snapshots-*******.op-*******.s3-outposts.us-west-2.amazonaws.com",
  "role_arn" : "arn:aws:iam::*******:role/elasticsearch-*******_role"
  }
}

Error response:

sdk_client_exception: Unable to execute HTTP request: es-snapshots---op-s3.es-snapshots-.op-*******.s3-outposts.us-west-2.amazonaws.com

{
  "type": "s3",
  "settings": {
  "bucket": "es-snapshots-*******--op-s3",
  "region" : "us-west-2",
  "endpoint" : "op-*******.s3-outposts.us-west-2.amazonaws.com",
  "role_arn" : "arn:aws:iam::*******:role/elasticsearch-*******_role"
  }
}

Error response:

amazon_s3_exception: The authorization header is malformed; Incorrect service s3. This endpoint belongs to s3-outposts.

{
  "type": "s3",
  "settings": {
  "bucket": "es-snapshots-*******--op-s3",
  "region" : "us-west-2",
  "endpoint" : "s3-outposts.us-west-2.amazonaws.com",
  "role_arn" : "arn:aws:iam::*******:role/elasticsearch-*******_role"
  }
}

Error response:

sdk_client_exception: Unable to execute HTTP request: es-snapshots-*******--op-s3.s3-outposts.us-west-2.amazonaws.com

{
  "type": "s3",
  "settings": {
  "bucket": "arn:aws:s3-outposts:us-west-2:*******:outpost/op-*******/accesspoint/es-snapshots",
  "region" : "us-west-2",
  "endpoint" : "es-snapshots-*******.op-*******.s3-outposts.us-west-2.amazonaws.com",
  "role_arn" : "arn:aws:iam::*******:role/elasticsearch-*******_role"
  }
}

Error response:

amazon_s3_exception: The authorization header is malformed; Incorrect service s3. This endpoint belongs to s3-outposts.

For example in normal region it is possible to register s3 repository by using AccessPoint alias without any problems:

{
  "type": "s3",
  "settings": {
  "bucket": "elastic-*******-s3alias",
  "endpoint": "s3.us-west-2.amazonaws.com",
  "region" : "us-west-2",
  "role_arn" : "arn:aws:iam::*******:role/elasticsearch-*******_role"
  }
}

It seems that S3 client is not capable of handling different endpoint even though it should.

Am I missing something or my case could be a feature proposition?

1 Like
2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.