Snapshots and malicious deletion of backups

Elastic Stack Elasticsearch
,
1

Hi there, I'm evaluating the options to protect against malicious deletion of backups.
The recommendation is usually to keep some of the backups in offline storage and give the backup process the minimum permission possible (usually write-only).

Since Elasticsearch snapshots are stored in the S3 bucket as a set of objects managed by the snapshot process, the backup process always needs full access to read/write/delete all objects in the bucket. In this scenario, an attacker with permission to edit the snapshot configuration is able to delete all backups.

Is there any best practice we can follow to keep backups protected from intended deletion? Amongst the possibilities I'm thinking are the use of S3 versioning (but I don't know if this will upset Elasticsearch), or maybe use some sort of independent replication process to copy the whole S3 bucket into a different one.

Thanks.

2

S3 versioning will not upset Elasticsearch because the "regular" APIs that it uses all hide deleted objects by default. However I think it will be tricky to recover from a malicious deletion attack using this approach. If you don't really delete anything then it's clearly theoretically possible to restore lost data, but I expect it'll be tricky in practice without good knowledge of the inner workings of the snapshot process and the repository data layout. There are some docs on this in the source code but this stuff is all implementation-defined, subject to change, and not something on which I'd recommend you rely.

My best advice is to take regular snapshot repository backups using some external process that you can trust.

2 Likes
3

Thanks @DavidTurner for pointing to the right docs, that's what I was looking for.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.