S3 repository empty

Elastic Stack Elasticsearch
1

Hello,

I am using cloud-aws plugin. I want to create a S3 repository , I have added the following lines in my elasticsearch.yml:

cluster.name : xx
node.name : xx
node.master: false
node.data: true
network.host: _ec2_
cloud.aws.access_key: xx
cloud.aws.secret_key: xx
discovery.type: ec2
plugin.mandatory: cloud-aws
repositories.s3.bucket: my-backup
repositories.s3.region: us-west-2

With the above configuration I am able to setup my cluster, but when I query my data node for S3 Repository it gives me empty.

I am using the following command

http://1.2.3.4:9200/_snapshot/_all

output : {}

Is it necessary to fire the following command first time in order to register/create a repository:

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "bucket": "my_bucket_name",
    "region": "us-west"
  }
}

My access key and secret key has full access for S3.

Thank you

2

Yes You have to PUT a repository as you did.

Is there a problem here?

3

Thanks dadoonet

I am executing the following command through my browser.

public_ip:9200/_snapshot/my
{"type":"s3","settings":{"bucket":"my-backup","region":"us-west-2"}}

I am facing the following error:

{
error: {
root_cause: [
{
type: "repository_missing_exception",
reason: "[my{"type":"s3"] missing"
}
],
type: "repository_missing_exception",
reason: "[my{"type":"s3"] missing"
},
status: 404
}

What may be the reason for this error.

4

The plugin is not loaded? May be you did not restart the nodes?

5

Sir I restarted all of my nodes .

Still I am facing same error. I hope I am using right command.

Secondly If I modify my command and If I add / after "my" as follows:

public_ip:9200/_snapshot/my/
{"type":"s3","settings":{"bucket":"my-backup","region":"us-west-2"}}

I am facing the following error :

{
error: {
root_cause: [
{
type: "invalid_snapshot_name_exception",
reason: "[my:{"type":"s3"] Invalid snapshot name [{"type":"s3"], must not contain the following characters [\, /, *, ?, ", <, >, |, , ,]"
}
],
type: "invalid_snapshot_name_exception",
reason: "[my:{"type":"s3"] Invalid snapshot name [{"type":"s3"], must not contain the following characters [\, /, *, ?, ", <, >, |, , ,]"
},
status: 400
}

Thank you

6

Done Sir,

I executed the following command on my node:

curl -XPOST "http://10.0.1.132:9200/_snapshot/my_backup" -d'
{
  "type": "s3",
  "settings": {
    "bucket": "my-backup",
    "region": "us-west-2"
  }
}
'

Now I can see my repository.

One more question Sir, will the plugin will take snapshots automatically or I will have to fire command each time for taking snapshot and restoring it.

If the plugin does not takes snapshot automatically , is there any other tool or third party software to take snapshot automatically .

Thank you.

7

No

I don't think so. But something like curator could help.
You could use a crontab to run it every x minutes if you wish.

8

Thank you Sir,