Snapshot multiple indexes on cold tier only

Elastic Stack Elasticsearch
, ,
1

Hello guys,

we have couple of our end users requesting to store their data in Elasticsearch for auditing purpose for 3 years. As data grows we have the bottleneck in our resources,
so we have decided snapshot the data from the cold tier (we do force merge and no replicas) and move to s3 storage before we actually perform delete in ILM policy.
We rollover indexes based on either size or days, if an indexed is rolled over we see index name with date extension,
eg: sampleindex-01-09-2023,sampleindex-02-09-2023,sampleindex-03-09-2023… etc.

Now the challenge here is, how can we take the snapshots of indexes which are in cold tier only as we have index names same in all tiers except data as postfix to the name.

is there a way we can have specific index names to each of the tiers(hot, warm, cold) during rollover or any other alternative to take those snapshots from cold tier?

any suggestions would be much appreciated.

regards and thank you.
Shubham

2

The round about way to do this would be to move indices to the cold tier, and then use an SLM policy before you use a delete phase in ILM via wait for snapshot.

But theres's no way to specifically snapshot only indices in a specific tier that I can see. It's an interesting idea though, and I would encourage you to raise a feature request on GitHub for either ILM or SLM to be able to provide that option.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.