# Delete some snapshots on S3 **URL:** \ **Category:** Elasticsearch\ **Created:** [October 8, 2018, 2:53pm UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482 "2018-10-08T14:53:15Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![dao](https://avatars.discourse-cdn.com/v4/letter/d/a6a055/32.png) [@dao](https://discuss.elastic.co/u/dao)\ **Post date:** [October 8, 2018, 2:53pm UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/1 "2018-10-08T14:53:15Z")
Hello, I have created a snapshot automation to s3 with a crontab `0 */3 * * * curl -XPUT -u elastic:aaaaaa "http://localhost:9200/_snapshot/s3_repository/snapshot_$(date +\%Y\%m\%d_\%H)"` I can see my S3 billing going up, and I want to cron the deletion of the old snapshots. I have started doing `curl -XDELETE -u elastic:aaaaa "http://localhost:9200/_snapshot/s3_repository/$old_snap` but it is veeery slow. What is the good practice to manage the snapshots? I just need a set of snapshots in case of crash or admin error. It means like 2 weeks retention best ---
**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Post date:** [October 9, 2018, 9:01am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/2 "2018-10-09T09:01:57Z")
@dao, You can use life cycle management of S3. In Life cycle management of S3 you can set the time period after how much time your data should be deleted from S3. Thanks. ---
**Author:** ![dao](https://avatars.discourse-cdn.com/v4/letter/d/a6a055/32.png) [@dao](https://discuss.elastic.co/u/dao)\ **Post date:** [October 9, 2018, 11:44am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/3 "2018-10-09T11:44:49Z")
Correct, but how do I know what file of the bucket I can delete? I imagine that some the files created by the snapshots are _linked_ in a certain way (I have understood that the snapshots are incremental)? If I delete a file, I may corrupt some snapshots, right? ---
**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Post date:** [October 9, 2018, 12:22pm UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/4 "2018-10-09T12:22:53Z")
@Dao, As far as i can tell we can delete any snapshots and it will not have any impact on other snapshot. You can use elasticsearch curator also to delete the snapshots. Please refer the below link for curator config: [https://discuss.elastic.co/t/deleting-old-snapshots/134085](https://discuss.elastic.co/t/deleting-old-snapshots/134085) Please refer the below link which saying that we can keep those snapshots which we want and delete the rest. [https://support.cloudbees.com/hc/en-us/articles/115000592472-Managing-snapshots-of-your-Elasticsearch-indices-](https://support.cloudbees.com/hc/en-us/articles/115000592472-Managing-snapshots-of-your-Elasticsearch-indices-) Thanks. ---
**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Post date:** [October 10, 2018, 3:23am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/5 "2018-10-10T03:23:02Z")
@Dao, As per my understanding, Indices are created with timestamp means one indices per day for single index pattern. So all the indices are independent to each other. You can delete any indices or snapshot and will not have any impact on the other indices or snapshots. Thanks. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 10, 2018, 6:31am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/6 "2018-10-10T06:31:33Z")
Do not ever delete files directly from a snapshot repository unless you are deleting the complete repository. Instead use [the APIs](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/modules-snapshots.html#_snapshot) to delete snapshots. [This blog post](https://www.elastic.co/blog/found-elasticsearch-snapshot-and-restore) is old, but still describes what goes on behind the scenes and how it works quite well. ---
**Author:** ![Tek\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tek_chand/32/34318_2.png) [@Tek\_Chand](https://discuss.elastic.co/u/Tek_Chand)\ **Post date:** [October 10, 2018, 11:12am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/7 "2018-10-10T11:12:46Z")
@Christian, Thank you for your response. > [@Christian\_Dahlqvist](#): > > Do not ever delete files directly from a snapshot repository unless you are deleting the complete repository. Yes... its will be quite difficult to identify the files for a specific snapshot. > [@Christian\_Dahlqvist](#): > > Instead use [the APIs](https://www.elastic.co/guide/en/elasticsearch/reference/6.4/modules-snapshots.html#_snapshot) to delete snapshots. As i have read that post and snapshots are incremental. So i have one question for you, **Can we delete any snapshot at any time using API or using curator?** Can we set life cycle on our S3 bucket so it will delete the snapshot from bucket? But as per my knowledge it will be like deleting a file directly from repository. So we should not use life cycle on S3 to delete the snapshot. Thanks. ---
**Author:** ![dao](https://avatars.discourse-cdn.com/v4/letter/d/a6a055/32.png) [@dao](https://discuss.elastic.co/u/dao)\ **Post date:** [October 11, 2018, 7:23am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/8 "2018-10-11T07:23:04Z")
Therefore, back to my initial question: deleting a snapshot is veeery slow (using the APIs) so, is there an alternate solution? ---
**Author:** ![dao](https://avatars.discourse-cdn.com/v4/letter/d/a6a055/32.png) [@dao](https://discuss.elastic.co/u/dao)\ **Post date:** [October 15, 2018, 7:01am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/9 "2018-10-15T07:01:09Z")
There is no solution, then? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 15, 2018, 7:05am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/10 "2018-10-15T07:05:51Z")
@Tek_Chand You can create any snapshot at any time. Remaining snapshots will make sure they have access to all segments required to report data as of that point in time. You should NOT delete files directly from a bucket as old segments may still be in use, which would potentially corrupt newer snapshots. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 15, 2018, 7:08am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/11 "2018-10-15T07:08:19Z")
@dao If you have a lot of snapshots there can be a lot of processing determining exactly which segments that need to be kept. You could perhaps set up a new repository and switch to this. Be sure to trim old snapshots regularly to keep the size down. This will also make it easier to process. Once you no longer need the old repository you can delete it completely. ---
**Author:** ![dao](https://avatars.discourse-cdn.com/v4/letter/d/a6a055/32.png) [@dao](https://discuss.elastic.co/u/dao)\ **Post date:** [October 15, 2018, 7:45am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/13 "2018-10-15T07:45:41Z")
OK, I'm moving to this process. here is the current setup, if it can help ```auto - name: Cron snapshots cron: name="elk-snapshots" minute="0" hour="*/3" job="curl -XPUT -u snap:aaaa \"http://localhost:9200/_snapshot/s3_bigdata_repo/snapshot_$(date +\%Y\%m\%d_\%H)\"" when: backup==True tags: snapshot - name: Delete 15-day-old cron snapshots cron: name="elk-snapshots-delete-old" minute="40" hour="*/3" job="curl -XDELETE -u snap:aaaa \"http://localhost:9200/_snapshot/s3_bigdata_repo/snapshot_$(date --date=\"15 day ago\" +\%Y\%m\%d_\%H)\"" when: backup==True tags: snapshot ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 12, 2018, 7:45am UTC](https://discuss.elastic.co/t/delete-some-snapshots-on-s3/151482/14 "2018-11-12T07:45:43Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.