The Cloud Aws plugin , overwrites the earlier snapshots in the bucket. Doesn't it impact restore operation? Or is this the expected behaviour? Also how do we purge snapshot older than x days, is purging files older than x days equivalent of it?
Every time you run a new snapshot, it copies the new "files" (new segments) to the the repository and basically link the snapshot with this set of files (existing ones and new ones).
If you remove a snapshot, it will look at all files and will remove the ones which are not used anymore by any of the remaining snapshots.
When you restore a snapshot, it will restore all the files linked to this snapshot.
You should never try to remove manually files in the repository but let the snapshot API do the right job for you.
HTH
What happens in the following Case:
-
some files(data/indices) are deleted from Elastic search.
-
Followed by compaction(optimize).
-
Followed by new snapshot.
Does the API ensure the removed files are retained in s3 bucket since they are being referred by earlier snapshots?
Yes. It does.
Old segments which contain old data are still referenced by a snapshot so they are kept around until you remove all the snapshots which refer to it.
Alright. Thanks for explaining it in detail.