S3 Repository

[Haytham_Shammout]

Hello Dears,

We are new to S3 repository and taking snapshots in ELK, we faced a case as below appreciate your support on it.

First of all we implement policy to take snapshots as below SC.

after taking 2 snapshots based on this policy we noticed that each snapshot contain all indices under dp_logs_default-* index pattern and this is not the behavior that we want we want to make it incrementally to take each snapshot for just 1 day after taking all days in the first snapshot.

[yago82]

@Haytham_Shammout

after taking 2 snapshots based on this policy we noticed that each snapshot contain all indices under dp_logs_default-* index pattern and this is not the behavior that we want we want to make it incrementally to take each snapshot for just 1 day after taking all days in the first snapshot.

HI,

Elasticsearch snapshots are incremental by default, meaning that each snapshot only stores data that is not part of an earlier snapshot.

If you want to reduce the amount of data that is backed up in each snapshot, you could consider using time-based indices (e.g., dp_logs_default-2023.01.01 , dp_logs_default-2023.01.02 , etc.), and then adjust your snapshot policy to only include the index for the current day.

Regards