Backup and Restore to GCS

Elastic Stack Elasticsearch
1

Hi, I've been tasked to create the backup and restore for our Elasticsearch. I want to create the backup on my local machine and then copy the snapshot to GCS. Google Cloud Storage. How can I accomplish this without creating the repository. We need to encrypt it before we copy to GCS. Anyone have any ideas?

2
  1. Create a Shared FS repository.
  2. Backup.
  3. Then encrypt your data and upload manually to GCS.

Would that work for you?

3

We created a path repo ``` [ root@curl-3381698604-qgdf5:/ ]$ curl -XPUT 'http://elasticsearch:9200/_snapshot/my_backup' -d '{

"type": "fs",
"settings": {
    "location": "/data/backups/my_backup",
    "compress": true

Add Comment Click to expand inline 8 lines

But when we try to create the backup we get

$ kubectl exec es-client-800761600-jvc98 -ti -- sh
/ # ls -l data
total 12
drwxr-xr-x 2 elastics elastics 4096 May 24 23:34 backups
drwxr-xr-x 3 elastics elastics 4096 May 24 23:34 data
drwxr-xr-x 2 elastics elastics 4096 May 24 23:34 log

/ # ls -l data/backups/
total 0

[ root@curl-3381698604-qgdf5:/ ]$ curl -XPUT 'http://elasticsearch:9200/_snapshot/my_backup' -d '{

"type": "fs",
"settings": {
    "location": "/mount/backups/my_backup",
    "compress": true
}

}'
{"error":{"root_cause":[{"type":"repository_exception","reason":"[my_backup] location [/mount/backups/my_backup] doesn't match any of the locations specified by path.repo because this setting is empty"}],"type":"repository_exception","reason":"[my_backup] failed to create repository","caused_by":{"type":"repository_exception","reason":"[my_backup] location [/mount/backups/my_backup] doesn't match any of the locations specified by path.repo because this setting is empty"}},"status":500}

4

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

You need to whitelist the directory in your config with path.repo setting.

5

our elasticsearch.yml has

path:
    data: /data/data
    logs: /data/log
    repo: /data/backups

$ curl -XPUT 'http://elasticsearch:9200/_snapshot/my_backup' -d '{
>     "type": "fs",
>     "settings": {
>         "location": "/data/backups/my_backup",
>         "compress": true
>     }
> }'
 {  
   "error":{  
  "root_cause":[  
     {  
        "type":"repository_verification_exception",
        "reason":"[my_backup] [[I-1gwDl4TTiP3wukksl2IA, 'RemoteTransportException[[es-data-2][10.84.1.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-2}{I-1gwDl4TTiP3wukksl2IA}{n47lzZ8pTAm9SWJ0PMK8Wg}{10.84.1.18}{10.84.1.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [B1c0QQ_lRAu_EK8N6xHO5A, 'RemoteTransportException[[es-data-1][10.84.3.23:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-1}{B1c0QQ_lRAu_EK8N6xHO5A}{AAZGTaTFSHGhAOEu3O9-Ng}{10.84.3.23}{10.84.3.23:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [rsKtAeB7TBCoXgQIvw9w7g, 'RemoteTransportException[[es-master-3164650403-vwtq0][10.84.2.16:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-vwtq0}{rsKtAeB7TBCoXgQIvw9w7g}{5PSio-OOTKaV5SFMKIV0Zw}{10.84.2.16}{10.84.2.16:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lQ92eZngTC-DCuJAysPsVg, 'RemoteTransportException[[es-master-3164650403-s4z63][10.84.1.17:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-s4z63}{lQ92eZngTC-DCuJAysPsVg}{_dt8gCY3SfKQIIClCgw19A}{10.84.1.17}{10.84.1.17:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lLXa-6tzRiS83XQKEWe8bg, 'RemoteTransportException[[es-data-0][10.84.2.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-0}{lLXa-6tzRiS83XQKEWe8bg}{jD5r6jgtS8iUUF9kOdo08Q}{10.84.2.18}{10.84.2.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];']]"
     }
  ],
  "type":"repository_verification_exception",
  "reason":"[my_backup] [[I-1gwDl4TTiP3wukksl2IA, 'RemoteTransportException[[es-data-2][10.84.1.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-2}{I-1gwDl4TTiP3wukksl2IA}{n47lzZ8pTAm9SWJ0PMK8Wg}{10.84.1.18}{10.84.1.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [B1c0QQ_lRAu_EK8N6xHO5A, 'RemoteTransportException[[es-data-1][10.84.3.23:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-1}{B1c0QQ_lRAu_EK8N6xHO5A}{AAZGTaTFSHGhAOEu3O9-Ng}{10.84.3.23}{10.84.3.23:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [rsKtAeB7TBCoXgQIvw9w7g, 'RemoteTransportException[[es-master-3164650403-vwtq0][10.84.2.16:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-vwtq0}{rsKtAeB7TBCoXgQIvw9w7g}{5PSio-OOTKaV5SFMKIV0Zw}{10.84.2.16}{10.84.2.16:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lQ92eZngTC-DCuJAysPsVg, 'RemoteTransportException[[es-master-3164650403-s4z63][10.84.1.17:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-s4z63}{lQ92eZngTC-DCuJAysPsVg}{_dt8gCY3SfKQIIClCgw19A}{10.84.1.17}{10.84.1.17:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lLXa-6tzRiS83XQKEWe8bg, 'RemoteTransportException[[es-data-0][10.84.2.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-0}{lLXa-6tzRiS83XQKEWe8bg}{jD5r6jgtS8iUUF9kOdo08Q}{10.84.2.18}{10.84.2.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];']]"
   },
   "status":500
}
6

The same dir must exist on every node and shared by all nodes

7

Thks

Well It does exist on every nodes

I'm not sure about the "shared by all nodes" part?

8

Can you run the following command on node 10.84.1.18:

ls -l /data/backups/my_backup
9

David,

I believe that Patrick has already submitted that. It’s probably in the thread below.

Suzanne

10

Was that run on node 10.84.1.18?

11

David,

Patrick asked “If repository-gcs plugin is tls/ssl able for data transit to an ecrypted gcs bucket?”

Suzanne

12

Sorry. I don't read that in the thread. But I'm on a mobile so I'm probably not seeing it. Can you link to this question please?

13

Let me know if you got this.

Suzanne

14

What?

15

David,

Patrick asked “if repository-gcs plugin is tls/ssl able for data transit to an ecrypted gcs bucket?”

Let me know if you got this.

Suzanne

16

I don't think he ever asked this. Can you tell me where he asked that?

17

No worries. Sorry, it might have been me when I replied. I took him off the email.

Suzanne

18

Sorry for hijacking the thread with another topic :slight_smile:

I've moved this specific plugin question here

19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.