How to archive snapshots

Elastic Stack Elasticsearch
1

Hi all,

We have a requirement to archive snapshots for long term retention. However since the snapshot directories are incremental, we do not want to simply zip it and send to archive storage. Is there a recommended alternate way to do this?

In short, we just want snapshots for last 7 days in the repository(which is in azure blob container) and want to send snapshots to archival storage (preferably onpremise) as they grow older than 7 days.

2

Yes, see these docs for more information.

3

Thanks for the response. I understand that the way to do it is to take backup of the repository contents. But its not clear how this can be done in a scenario, where we want to archive one snapshot per day (and not a folder with a bunch of snapshots).

    "policy" : {
      "name" : "<auto_snapshot-{now{yyyy-MM-dd_HH-mm-ss}}>",
      "schedule" : "0 15 */6 * * ?",
      "repository" : "snapshots",
      "config" : {
        "feature_states" : [
          "watcher",
          "fleet",
          "transform",
          "logstash_management",
          "machine_learning",
          "enrich",
          "kibana",
          "tasks"
        ],
        "include_global_state" : true,
        "indices" : "*,-.*"
      },
      "retention" : {
        "expire_after" : "7d",
        "min_count" : 10,
        "max_count" : 20
      }

Should we define another repository and SLM policy that takes single snapshots per day and retains a single snapshot - and then archive the contents of that directory every day ?

4

Archiving a single snapshot doesn't really make sense, or at least it will be very expensive because it loses all the benefits of cross-snapshot data deduplication.

5

The intention here is to have a recovery point in archive for every day, lets say for an year. Could you suggest any strategies?

6

Yes, use the backup process I linked previously, combined with an archival process that handles incremental data properly. I.e. not this:

simply zip it and send to archive storage

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.