Create snapshot on on-prem S3

Elastic Stack Elasticsearch
1

Hello,
I have problem with integration with snapshot. I have on-prem s3 and when I tried to create repository I have a error: Unknown s3 client name [test]. Existing client configs: default.

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "logs",
    "client": "test",
    "endpoint": "10.41.131.12",
    "protocol": "https"
  }
}

And response:

{
  "error": {
    "root_cause": [
      {
        "type": "repository_exception",
        "reason": "[my_s3_repository] cannot create blob store"
      }
    ],
    "type": "repository_verification_exception",
    "reason": "[my_s3_repository] path  is not accessible on master node",
    "caused_by": {
      "type": "repository_exception",
      "reason": "[my_s3_repository] cannot create blob store",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "Unknown s3 client name [test]. Existing client configs: default"
      }
    }
  },
  "status": 500
}

In elasticsearch.yml I added

s3.client.logisnapshoty-user.endpoint: 10.41.131.12:443

and also access key and secret key to keystore.

Could you help me, add external own s3 repository?

2

And what is the client named you used? The log you shared mentions that the test client does not exist, only the default client.

You need to use the correct client as mentioned in the documentation.

So if in your keystore you added s3.client.default.access_key and s3.client.default.secret_key, then the client name is default, not test.

If you want to use test as the client name you need to add the access_key and secret_key for this client as well, like this:

bin/elasticsearch-keystore add s3.client.test.access_key
bin/elasticsearch-keystore add s3.client.test.secret_key
3

Same thing as mentioned before, the client needs to be the same for all settings.

The settings follows this naming scheme:

s3.client.CLIENT_NAME.SETTING_NAME

So if you added s3.client.logisnapshoty-user.endpoint in elasticsearch.yml you need to add s3.client.logisnapshoty-user.access_key and s3.client.logisnapshoty-user.secret_key

4

Thanks for your anserw. I changed my config and also have the same problem:

In my configuration elastisearch.yml

s3.client.logisnapshoty-user.endpoint: 10.71.151.2:443

In my keystore I added:

/usr/share/elasticsearch/bin/elasticsearch-keystore add s3.client.logisnapshoty-user.access_key

and

/usr/share/elasticsearch/bin/elasticsearch-keystore add s3.client.logisnapshoty-user.secret_key

and after send request:

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "logi-snapshoty",
    "client": "logisnapshoty-user",
    "endpoint": "10.71.151.2",
    "protocol": "https"
  }
}

and response:

{
  "error": {
    "root_cause": [
      {
        "type": "repository_exception",
        "reason": "[my_s3_repository] cannot create blob store"
      }
    ],
    "type": "repository_exception",
    "reason": "[my_s3_repository] Could not determine repository generation from root blobs",
    "caused_by": {
      "type": "repository_exception",
      "reason": "[my_s3_repository] cannot create blob store",
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "Unknown s3 client name [logisnapshoty-user]. Existing client configs: default"
      }
    }
  },
  "status": 500
}

I have 3 node cluster, and I execute this command and change elasticsearch.yml on master node, is it ok?

5

The snapshot configuration needs to be done on all master and data nodes, also any changes to elasticsearch.yml requires a restart.

So you need to change elasticsearch.yml on all nodes and restart them.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.