# Snapshots to s3; file TTL **URL:** \ **Category:** Elasticsearch\ **Created:** [May 8, 2016, 6:53pm UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492 "2016-05-08T18:53:45Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![sirkubax\_1](https://avatars.discourse-cdn.com/v4/letter/s/9de053/32.png) [@sirkubax\_1](https://discuss.elastic.co/u/sirkubax_1)\ **Post date:** [May 8, 2016, 6:53pm UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/1 "2016-05-08T18:53:45Z")
Hi I'm creating snapshot to S3 In my bucket I set TTL to 365 days, so the old files would be removed after 1 year. via > **[Snapshot module | Elasticsearch Guide \[8.11\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html)** > The index snapshot process is incremental. In the process of making the index snapshot Elasticsearch analyses the list of the index files that are already stored in the repository and copies only files that were created or changed since the last snapshot. I'm afraid, if I'd be able to restore 'most recent' (younger that 1 year) snapshot, when the older files would be deleted. Have You been testing, would the mechanism work, if some 1-year-old files would vanish? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 8, 2016, 8:55pm UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/2 "2016-05-08T20:55:29Z")
> [@sirkubax\_1](#): > > would the mechanism work, if some 1-year-old files would vanish? Yes. The process will only delete files if they are not related to older snapshots. ---
**Author:** ![sirkubax\_1](https://avatars.discourse-cdn.com/v4/letter/s/9de053/32.png) [@sirkubax\_1](https://discuss.elastic.co/u/sirkubax_1)\ **Post date:** [May 9, 2016, 7:46am UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/3 "2016-05-09T07:46:04Z")
Not sure You I was clear. It is not the Elasticsearch s3 plugin that is going to remove the files The files have Time-To-Live=365 days and they would be removed by the S3 filesystem (automatically - no process involved), regardless if they are related to other snapshot or not. that is why I need to check if the snapshot can restore 'it's part' from it's data. It is important, to be able to restore 'today's snapshot' without 'yesterday files' ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 9, 2016, 7:46am UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/4 "2016-05-09T07:46:49Z")
Right, then that won't work. S3 will potentially destroy your snapshots. ---
**Author:** ![sirkubax\_1](https://avatars.discourse-cdn.com/v4/letter/s/9de053/32.png) [@sirkubax\_1](https://discuss.elastic.co/u/sirkubax_1)\ **Post date:** [May 9, 2016, 8:29am UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/5 "2016-05-09T08:29:56Z")
Well, You should not give up so easily 🙂 I've just look around, probably changing the S3 TTL (life-cycle) rule to auto-remove files matching pattern only (like _indices/logstash-201\*_) should solve the problem. I guess we do care about index, metadata\* snapshot\* files, but the indices/\* that are not a part of a snapshot that we are restoring, could be missing, and still it would succeed. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 9, 2016, 10:16am UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/6 "2016-05-09T10:16:36Z")
It won't work. You need to delete the snapshots via the API. ---
**Author:** ![sirkubax\_1](https://avatars.discourse-cdn.com/v4/letter/s/9de053/32.png) [@sirkubax\_1](https://discuss.elastic.co/u/sirkubax_1)\ **Post date:** [May 9, 2016, 10:09pm UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/7 "2016-05-09T22:09:59Z")
Hi I'm happy, because my guess was correct. Please take a look: I do create a snapshot: `curl -XPUT "localhost:9200/_snapshot/my_fs_repository/2016.05.09_evening?wait_for_completion=true"` Do remove a index (2016.05.01) `curl -XDELETE 'http://localhost:9200/logstash-2016.05.01*?pretty'` Take another snapshot: `curl -XPUT "localhost:9200/_snapshot/my_fs_repository/2016.05.09_evening2?wait_for_completion=true"` Pretend to remove the index from filesystem snapshot (not via API) `mv /mnt/nfs/indices/logstash-2016.05.01/ /mnt/nfs/indices/logstash-2016.05.01_pseudoremove` The index content: `ls /mnt/nfs/indices/logstash-2016.05.01_pseudoremove 0 1 2 3 4 snapshot-2016.05.08 snapshot-2016.05.08_clean snapshot-2016.05.08_clean2 snapshot-2016.05.08_clean3 snapshot-2016.05.09_evening` `curl -XPOST "localhost:9200/*/_close"` #do I always have to close indices on 'full snapshot restore?' {"acknowledged":true} The first snapshot restore would fail `curl -XPOST "localhost:9200/_snapshot/my_fs_repository/2016.05.09_evening/_restore?pretty"` { "error" : "SnapshotException[[my\_fs\_repository:2016.05.09\_evening] failed to read metadata]; nested: FileNotFoundException[/mnt/nfs/indices/logstash-2016.05.01/snapshot-2016.05.09\_evening (No such file or directory)]; ", "status" : 500 } **But this one works** `curl -XPOST "localhost:9200/_snapshot/my_fs_repository/2016.05.09_evening2/_restore?pretty"` { "accepted" : true } **🙂** So with right file-delete policy it works without API delete call. Probably some snapshot-metadata may be left as orphans - but this is something I do accept. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:52pm UTC](https://discuss.elastic.co/t/snapshots-to-s3-file-ttl/49492/8 "2017-07-05T22:52:56Z")