ES Snapshot Delete

Elastic Stack Elasticsearch
1

Elasticsearch backups are always incremental. Let's say I have a daily backup job. After one week I have 7 snapshots.

On the 8th day if Im going to delete the day1's snapshot, then the rest of the snapshots will become useless? (since they are smaller in size.

Or before deleting the unchanged data will push it to the next incremental backup and make it a complete backup?

Another doubt is, if I want to restore the day 3's backup then will it only restore the day 3's backup(I mean that incremental data only) ? or apply snapshots from day 1 to day 3?

2

Each snapshot is a full snapshot of the data at that time but segments that have not changed since previous snapshot are not copied again which is what is often referred to as incremental (it is not in the traditional sense). You can delete any snapshot without compromising others. Any segments referred to by multiple snapshots will be kept until no longer needed.

3

I recently wrote a blog on this this works, as I had the same questions; it's a VERY nice system (though you should prune to keep total # under control as it has to read all the metadata and can get slow on big systems):

4

This old blog post also contains a quite good description of how it works.

5

Ah, that is nice, and I will link to it, as I thought I looked far & wide before writing one. It's more low-level than I was shooting for, as I tried to balance high-level operations with a bit of detail, but staying out of file contents. But nice map in there of snaps to segments :slight_smile:

Note the first image has a mistake in its caption which says "Indexes and their segments" but is actually of indices and their shards, as it only shows a single shard of each index, number 0 (as also shown in the later file list which is correct).

6

Hey Steve,

Its a very nice blog, thanks and its useful.

1 Like
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.