Backup strategy

Hey, I’m looking for some answers in terms of backup strategy in ES

I have a daily index that is being updated 2 times every hour, once at the half of the hour, and once at the end. (Each update is around 200k docs).
I want to back up those indices, every 30m (Basically after every update).
I am also forcemerging the index every 30m.

Will an incremental backup work in this case? Or because of the forcemerge it will run a full backup again.
Also, because I'm merging the segments, won't it causes segments that were backed up in the half point to be overwritten at the end?

1 Like

No, frocemerge merges existing segments into new segments, backup is segment-based, so it will have to copy over all new segments. So, if you merge everything into a single segment it's a full backup every time.

When backup procedure "locks" a list of segments that will be part of the backup at the start of the backup and prevents these segment from being deleted until the backup finishes. So the forcemerge that you start after backup starts will have no effect on what will be copied.

No, frocemerge merges existing segments into new segments, backup is segment-based, so it will have to copy over all new segments. So, if you merge everything into a single segment it's a full backup every time.

So the new segments will have different names/ids from the ones that were before the merge?
This means that if I have an index with 5 segments, and I backup them, I will have 5 segments in the snapshot.
If I then forcemerge that index & backup it again, I should see 6 segments in the backup repository, 5 from the first segment and 1 from after the merge.
Correct?

Yes, you will see all 6 segments, assuming that you did not delete the first snapshot. More over the 6th segment is going to have an approximately the same size as the first 5 combined.

1 Like

Thanks a lot. Thats what I wanted to know

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.