Question about Snapshot and Restore

Elastic Stack Elasticsearch
1

Suppose we have two cluster : A and B

Data from A is getting synced to B every 12 hour. So incremental snapshots from A are getting restored on B.

Suppose on a day, at first 12 hr, index1 was created with 10 docs. Now snapshot was taken and restored to cluster B.

Now on cluster B we have index1 with 10 docs. Suppose I index a document in index1 on cluster B. Now index1 on cluster B has 11 docs. 1 doc is new and present only in cluster B. Cluster A index 1 has 10 docs only.

Now, in next 12 hr, 10 more docs come to index1 in cluster A. So cluster A has now 20 docs. Incremental snapshot is taken and in this snapshot we have 10 new docs of index1.

Now if this incremental snapshot is restored in cluster B will it delete the 11th document which was indexed manually or will 10 new docs will be indexed making the count of index1 in cluster B to 21 docs??

Note: Cluster A index1 has 20 docs now

2

You cannot restore over the top of an existing index. So the process would be;

  • Snapshot on A
  • Delete on B
  • Restore on B
3

All snapshots are full snapshots of the data in the index so e we hen you restore the snapshot in cluster B it will be exactly as it was when taken on cluster A. Any data written to the index in cluster B will be lost. The part that is often referred to as incremental is the mechanism behind the scenes. As segments are immutable Ones that have not changed can be shared between different snapshots which mean all data is not necessarily copied for ever snapshot.

1 Like
4

What is the process by which elasticsearch snapshot captures only the changed segments? I mean how does it knows to pick only the changed data?

5

When a snapshot is created (segments are immutable so do not change), only new segments are copied to the repository and existing ones are marked for reuse. When a snapshot is restored all segments belonging to that snapshot, irrespective of when they were copied to the repository, are restored.

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.