Elastic Cloud warns when changing managed snapshot policy in Kibana

Elastic Stack Kibana
1

We have an Elastic Cloud instance which uses the default "cloud-snapshot-policy" for back-ups. We had a separate service running in GCP which manages data retention which was misconfigured and trimmed 9 months of data which was supposed to be kept.

When we went to use the restore option in Kibana we noticed that back-ups are only kept for ~72hrs, meaning we have lost that data forever.

Our concern with this default is if something goes wrong Friday evening and it doesn't get picked up until Tuesday (say it's a bank holiday weekend) it will be too late.

Under Stack Management > Data > Snapshot and Restore > Polices, I attempted to edit the "cloud-snapshot-policy" and change the back-up cron schedule to every hour (0 0 */1 * * ?) and under retention, change the "Delete After" to 3 days and max. count to 200. As I do so, I get the following warning:

(!) This is a managed policy. Changing this policy might affect other systems that use it. Proceed with caution.

As I understand it, this policy is therefore managed by the Elastic Cloud infrastructure and I shouldn't change it? Given the aforementioned logic, how do we satisfy our risk in losing data? Additionally, does anyone have any experience in running Elastic Cloud with a much longer data retention period? Anything we should know/consider?

Thanks in advance.

2

Hi Tom,

The warning you're seeing is because the "cloud-snapshot-policy" is a managed policy, meaning it's used by the Elastic Cloud infrastructure. Modifying it could potentially affect other systems that use it.

However, you can create a new snapshot policy that suits your needs.

Regards

3

Hi @yago82,

Sure but having a separate policy would presumably create another duplication of the data and therefore eat into our storage capacity unnecessarily?

4

Hi,

You're correct that creating a new snapshot policy would create additional snapshots, which could increase your storage usage. However, it's important to note that Elasticsearch snapshots are incremental. This means that each snapshot only stores data that has changed since the last snapshot.

However, I believe there shouldn't be significant issues with increasing the retention period from 3 to 7 days, for example.

Regards

1 Like
5

Ok thanks. That sounds like it may be the best option.