Periodically over the last year or two, I have taken snapshots of different indices. Some of these I have recorded in a spreadsheet, some I have not. Now, I would like to delete all snapshots older then 90 days that do not include the prefix 'citydirectory', but without having to go through the spreadsheet and deleting them one by one. Therefore by these requirements: a snapshot that's 6 months old called 'citydirectory_v1' would not be deleted, but another snapshot 'example2' that's also 6 months old would be deleted. What is the best way to do something like this? I was thinking of writing a python script that returns all snapshots on s3, but this may not include the dates. I also think there is an intuitive was to do this in Elasticsearch, or as part of the ElasticStack, although I can't remember what it is. If you think of a better way to do this, please let me know. Thank you.
you could try Elastic Curators delete Snapshot funtionality
Hey @Shaoranlaos, I'm taking a look at curator now. I will post my results here later. Thanks for the tip!
Hi @Shaoranlaos I have setup curator and have my script -- it's show below:
- 'a data node'
blacklist: ['elasticsearch', 'urllib3']
Delete snapshots from the selected repository older than 100 days
(based on creation_date), for everything but 'citydirectory-' prefixed snapshots.
- filtertype: pattern
- filtertype: age
For some reason I keep getting the exception 'Failed to complete action: delete_snapshots. <class 'curator.exceptions.FailedExecution'>: Unable to verify existence of repository dcs-elastic-snapshot
'. Any idea why it's not finding the correct s3 repository?
I figured it out. In repository - put the name you registered the s3 bucket as, not as what it appears in s3 if they differ. So for me I registered my s3 bucket under 'dcselasticsnapshot' rather then what is appears in s3 'dcs-elastic-snapshot'.
fyi this solves my question, so marking this as the correct answer.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.