Backup and Restore to GCS


(Suzanne) #1

Hi, I've been tasked to create the backup and restore for our Elasticsearch. I want to create the backup on my local machine and then copy the snapshot to GCS. Google Cloud Storage. How can I accomplish this without creating the repository. We need to encrypt it before we copy to GCS. Anyone have any ideas?


(David Pilato) #2
  1. Create a Shared FS repository.
  2. Backup.
  3. Then encrypt your data and upload manually to GCS.

Would that work for you?


(Suzanne) #3

We created a path repo ``` [ root@curl-3381698604-qgdf5:/ ]$ curl -XPUT 'http://elasticsearch:9200/_snapshot/my_backup' -d '{

"type": "fs",
"settings": {
    "location": "/data/backups/my_backup",
    "compress": true

Add Comment Click to expand inline 8 lines

But when we try to create the backup we get

$ kubectl exec es-client-800761600-jvc98 -ti -- sh
/ # ls -l data
total 12
drwxr-xr-x 2 elastics elastics 4096 May 24 23:34 backups
drwxr-xr-x 3 elastics elastics 4096 May 24 23:34 data
drwxr-xr-x 2 elastics elastics 4096 May 24 23:34 log

/ # ls -l data/backups/
total 0

[ root@curl-3381698604-qgdf5:/ ]$ curl -XPUT 'http://elasticsearch:9200/_snapshot/my_backup' -d '{

"type": "fs",
"settings": {
    "location": "/mount/backups/my_backup",
    "compress": true
}

}'
{"error":{"root_cause":[{"type":"repository_exception","reason":"[my_backup] location [/mount/backups/my_backup] doesn't match any of the locations specified by path.repo because this setting is empty"}],"type":"repository_exception","reason":"[my_backup] failed to create repository","caused_by":{"type":"repository_exception","reason":"[my_backup] location [/mount/backups/my_backup] doesn't match any of the locations specified by path.repo because this setting is empty"}},"status":500}


(David Pilato) #4

Please format your code using </> icon as explained in this guide. It will make your post more readable.

Or use markdown style like:

```
CODE
```

You need to whitelist the directory in your config with path.repo setting.


#5

our elasticsearch.yml has

path:
    data: /data/data
    logs: /data/log
    repo: /data/backups

$ curl -XPUT 'http://elasticsearch:9200/_snapshot/my_backup' -d '{
>     "type": "fs",
>     "settings": {
>         "location": "/data/backups/my_backup",
>         "compress": true
>     }
> }'
 {  
   "error":{  
  "root_cause":[  
     {  
        "type":"repository_verification_exception",
        "reason":"[my_backup] [[I-1gwDl4TTiP3wukksl2IA, 'RemoteTransportException[[es-data-2][10.84.1.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-2}{I-1gwDl4TTiP3wukksl2IA}{n47lzZ8pTAm9SWJ0PMK8Wg}{10.84.1.18}{10.84.1.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [B1c0QQ_lRAu_EK8N6xHO5A, 'RemoteTransportException[[es-data-1][10.84.3.23:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-1}{B1c0QQ_lRAu_EK8N6xHO5A}{AAZGTaTFSHGhAOEu3O9-Ng}{10.84.3.23}{10.84.3.23:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [rsKtAeB7TBCoXgQIvw9w7g, 'RemoteTransportException[[es-master-3164650403-vwtq0][10.84.2.16:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-vwtq0}{rsKtAeB7TBCoXgQIvw9w7g}{5PSio-OOTKaV5SFMKIV0Zw}{10.84.2.16}{10.84.2.16:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lQ92eZngTC-DCuJAysPsVg, 'RemoteTransportException[[es-master-3164650403-s4z63][10.84.1.17:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-s4z63}{lQ92eZngTC-DCuJAysPsVg}{_dt8gCY3SfKQIIClCgw19A}{10.84.1.17}{10.84.1.17:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lLXa-6tzRiS83XQKEWe8bg, 'RemoteTransportException[[es-data-0][10.84.2.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-0}{lLXa-6tzRiS83XQKEWe8bg}{jD5r6jgtS8iUUF9kOdo08Q}{10.84.2.18}{10.84.2.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];']]"
     }
  ],
  "type":"repository_verification_exception",
  "reason":"[my_backup] [[I-1gwDl4TTiP3wukksl2IA, 'RemoteTransportException[[es-data-2][10.84.1.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-2}{I-1gwDl4TTiP3wukksl2IA}{n47lzZ8pTAm9SWJ0PMK8Wg}{10.84.1.18}{10.84.1.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [B1c0QQ_lRAu_EK8N6xHO5A, 'RemoteTransportException[[es-data-1][10.84.3.23:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-1}{B1c0QQ_lRAu_EK8N6xHO5A}{AAZGTaTFSHGhAOEu3O9-Ng}{10.84.3.23}{10.84.3.23:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [rsKtAeB7TBCoXgQIvw9w7g, 'RemoteTransportException[[es-master-3164650403-vwtq0][10.84.2.16:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-vwtq0}{rsKtAeB7TBCoXgQIvw9w7g}{5PSio-OOTKaV5SFMKIV0Zw}{10.84.2.16}{10.84.2.16:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lQ92eZngTC-DCuJAysPsVg, 'RemoteTransportException[[es-master-3164650403-s4z63][10.84.1.17:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-master-3164650403-s4z63}{lQ92eZngTC-DCuJAysPsVg}{_dt8gCY3SfKQIIClCgw19A}{10.84.1.17}{10.84.1.17:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [lLXa-6tzRiS83XQKEWe8bg, 'RemoteTransportException[[es-data-0][10.84.2.18:9300][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[my_backup] a file written by master to the store [/data/backups/my_backup] cannot be accessed on the node [{es-data-0}{lLXa-6tzRiS83XQKEWe8bg}{jD5r6jgtS8iUUF9kOdo08Q}{10.84.2.18}{10.84.2.18:9300}]. This might indicate that the store [/data/backups/my_backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];']]"
   },
   "status":500
}

(David Pilato) #6

The same dir must exist on every node and shared by all nodes


#7

Thks

Well It does exist on every nodes

I'm not sure about the "shared by all nodes" part?


(David Pilato) #8

Can you run the following command on node 10.84.1.18:

ls -l /data/backups/my_backup

(Suzanne) #9

David,

I believe that Patrick has already submitted that. It’s probably in the thread below.

Suzanne


(David Pilato) #10

Was that run on node 10.84.1.18?


(Suzanne) #11

David,

Patrick asked “If repository-gcs plugin is tls/ssl able for data transit to an ecrypted gcs bucket?”

Suzanne


(David Pilato) #12

Sorry. I don't read that in the thread. But I'm on a mobile so I'm probably not seeing it. Can you link to this question please?


(Suzanne) #13

Let me know if you got this.

Suzanne


(David Pilato) #14

What?


(Suzanne) #15

David,

Patrick asked “if repository-gcs plugin is tls/ssl able for data transit to an ecrypted gcs bucket?”

Let me know if you got this.

Suzanne


(David Pilato) #16

I don't think he ever asked this. Can you tell me where he asked that?


(Suzanne) #17

No worries. Sorry, it might have been me when I replied. I took him off the email.

Suzanne


#18

Sorry for hijacking the thread with another topic :slight_smile:

I've moved this specific plugin question here


(system) #19

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.