Elasticsearch OCI bucket

[Walter_Hiranpat1 (Walter Hiranpat)]

Has there been any support for using the oci (oracle cloud infrastructure) bucket to perform snapshot and restore? I thought oci bucket uses similar s3 configuration and would be compatible to use with the aws s3 plugin but it doesn't seem to work for me.

I have generated access and security keys and store on the elasticsearch nodes. I was able to curl the bucket from the elasticsearch instances so they are reachable.

When I try to use verification status on the kibana page, it looks like it is going to "sdk_client_exception: The requested metadata is not found at http://.,./latest/meta-data/iam/security-credentials/" which looks like it might be referring to AWS endpoint.

I am assuming that the aws s3 plugin isn't compatible with the oci bucket configuration, is that correct? Is there a solution to allow me to snapshot to an oci bucket storage?

[leandrojmp (Leandro Pereira)]

Are you using Elasticsearch on Elastic Cloud or self-hosted?

If I'm not wrong, Elasticsearch on Elastic Cloud only supports AWS S3, but the self-hosted version supports S3-compatible services .

Never used OCI buckets and I'm not sure if it is compatible with S3, but if they are, you will need to change the endpoint setting while creating the repository and pass the endpoint of the bucket API in OCI.

From the documentation:

There are a number of storage systems that provide an S3-compatible API, and the repository-s3 type allows you to use these systems in place of AWS S3. To do so, you should set the s3.client.CLIENT_NAME.endpoint setting to the system’s endpoint.

[Walter_Hiranpat1 (Walter Hiranpat)]

We are using self-hosted version of elasticsearch and I try configuring it through the kibana ui.

I have also configured these field in elasticsearch.yml: s3.client.default.endpoint, s3.client.default.protocol

In one the error message reason says that:

<base path> is not accessible on master node

As I look lower in the error message there is a reason field:

"amazon_s3_exception: Unauthorized (Service: Amazon S3; Status Code: 401; Error Code: 401 Unauthorized; Request ID: null; S3 Extended Request ID: null)"

But I have set the access_key, secret_key and session_token through the

elasticsearch-keystore add <commands>

[leandrojmp (Leandro Pereira)]

What endpoint did you add?

In elasticsearch.yml you just need to add s3.client.default.endpoint, there is no need to add s3.client.default.protocol because the default is already https.

From Oracle documentation the endpoint will be something like:

<object_storage_namespace>.compat.objectstorage.<location>.oraclecloud.com

You will need to check how to get the object_storage_namespace value for your account and the location will also depend on your account, this needed to be check within Oracle documentation.

After you set it in all your nodes and restart them, you may then add the repository and the values in the keystore.