Snapshot & Restore clarification

Elastic Stack Elasticsearch
1

Hi all,

We are using Elasticsearch 1.7.2 in Prod and having 3 master nodes, 3 data nodes & 1 client node. ES cluster has 3 indices such as,

Index name Primary size Shards
es_chr, 2 GB 1
es_cval 173 GB 6
es_item 850 GB 18

Yesterday we took snapshot in Production cluster and I can see in repository snapshot files/folders were generated with based on the number of shards allotted and then we restored the indices in DR(Di aster Recovery) cluster for backup purpose. Restoring in DR cluster, I manually moved the snapshot files from PROD to DR server (through SCP) and I started restoring it and process were completed well. Suppose today If i am going to take snapshot of index es_item, data might be changed or new data will be added in index.

  1. Should I again manually need to move snapshot files from PROD to DR server for backup?
    (or)

  2. Is there any other workaround to move incremental data?

  3. How to find the difference between initial & incremental snapshot?

Please kindly help me in this..

Thanks,
Ganeshbabu R

2

If I get it correctly, I'd rsync the "repository" in PROD DC to DR DC.

I would not try to copy "only" a snapshot. Snapshot files are shared within the same repository.

3

Thanks for your response @dadoonet

We don't have a common repository for PROD & DR server. For PROD server we created NAS mount for repository to take backup but for the DR server location were different so that we can't rsync the repository to DR cluster. So we created local repository in DR server and manually moving snapshot files from PROD repository to DR local repository.

Please give your feedback to overcome this problem.

Thanks,
Ganeshbabu R

4

I don't know. I'm not a sys guy.

But you need to add all files available in the PROD repository to the DR repository.
I don't know how you can "diff" the two directories... May be sorting by date and only copy the "new" files?

5

Thanks for your response @dadoonet

6

I took second snapshot after adding some files and try to find the difference. I found that some files also got removed and some got modified. But if I am adding documents, why some files get removed from the backup?

7

Please start a new thread for your question.

8

Here is the new thread as asked.