# \[ES-1.4.2\] Snapshot FS Repository directory structure **URL:** \ **Category:** Elasticsearch\ **Created:** [February 16, 2015, 8:04am UTC](https://discuss.elastic.co/t/es-1-4-2-snapshot-fs-repository-directory-structure/22181 "2015-02-16T08:04:41Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Yarden\_Bar\_2](https://avatars.discourse-cdn.com/v4/letter/y/ed655f/32.png) [@Yarden\_Bar\_2](https://discuss.elastic.co/u/Yarden_Bar_2)\ **Post date:** [February 16, 2015, 8:04am UTC](https://discuss.elastic.co/t/es-1-4-2-snapshot-fs-repository-directory-structure/22181/1 "2015-02-16T08:04:41Z")
Hi All, We're using the snapshot API to backup our ES cluster and as time advances, the repository storage utilization is growing. Currently I use 'rsync' to copy snapshot repository data outside of our production environment, but eventually I'd need to delete some snapshots to free space for new ones. The repository keeps files like 'index', 'snapshot-\*' and 'metadata-' files and I suspect that deleting these files will fail the repository from being functional. Is there a way that i can 'rsync' the snapshot repository to another location and delete the existing snapshots? Can I delete that snapshot data itself (the 'indices' directory the resides in the repository) without damaging the repository itself? Thanks for any idea, Yarden -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 16, 2015, 9:29am UTC](https://discuss.elastic.co/t/es-1-4-2-snapshot-fs-repository-directory-structure/22181/2 "2015-02-16T09:29:47Z")
In these sorts of cases it may make sense to have a repo for a week/months worth of snapshots, then just retain however much you need in these rolling repos. Kind of like what LS does for its indices. Regarding deleting the files, take a look at > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. On 16 February 2015 at 19:04, Yarden Bar [yarden@astoncapitalmanagement.com](mailto:yarden@astoncapitalmanagement.com) wrote: > Hi All, > We're using the snapshot API to backup our ES cluster and as time > advances, the repository storage utilization is growing. > Currently I use 'rsync' to copy snapshot repository data outside of our > production environment, but eventually I'd need to delete some snapshots to > free space for new ones. > > The repository keeps files like 'index', 'snapshot-\*' and 'metadata-' > files and I suspect that deleting these files will fail the repository from > being functional. > > Is there a way that i can 'rsync' the snapshot repository to another > location and delete the existing snapshots? > Can I delete that snapshot data itself (the 'indices' directory the > resides in the repository) without damaging the repository itself? > > Thanks for any idea, > Yarden > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_gCbDhBwyGWRrHR42REy\_4pti60psX73ue9Z5rKSC3Dw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_gCbDhBwyGWRrHR42REy_4pti60psX73ue9Z5rKSC3Dw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Yarden\_Bar\_2](https://avatars.discourse-cdn.com/v4/letter/y/ed655f/32.png) [@Yarden\_Bar\_2](https://discuss.elastic.co/u/Yarden_Bar_2)\ **Post date:** [February 16, 2015, 9:43am UTC](https://discuss.elastic.co/t/es-1-4-2-snapshot-fs-repository-directory-structure/22181/3 "2015-02-16T09:43:36Z")
If I understand you correctly, the monthly/weekly repository will get purged when we decide to, right? Another solution crossed my mind: Thinking of HDFS, which I'm not so familiar with, but is there some kind of HDFS node which is similar to ES node which is not a master nor a data? This way all snapshot will go through a HDFS 'gateway' node to outside of our prod env. On Monday, February 16, 2015 at 11:30:14 AM UTC+2, Mark Walkom wrote: > In these sorts of cases it may make sense to have a repo for a week/months > worth of snapshots, then just retain however much you need in these rolling > repos. Kind of like what LS does for its indices. > > Regarding deleting the files, take a look at > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/backing-up-your-cluster.html#_deleting_snapshots) > > On 16 February 2015 at 19:04, Yarden Bar \< > [yar...@astoncapitalmanagement.com](mailto:yar...@astoncapitalmanagement.com) \\> wrote: > > > Hi All, > > We're using the snapshot API to backup our ES cluster and as time > > advances, the repository storage utilization is growing. > > Currently I use 'rsync' to copy snapshot repository data outside of our > > production environment, but eventually I'd need to delete some snapshots to > > free space for new ones. > > > > The repository keeps files like 'index', 'snapshot-\*' and 'metadata-' > > files and I suspect that deleting these files will fail the repository from > > being functional. > > > > Is there a way that i can 'rsync' the snapshot repository to another > > location and delete the existing snapshots? > > Can I delete that snapshot data itself (the 'indices' directory the > > resides in the repository) without damaging the repository itself? > > > > Thanks for any idea, > > Yarden > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0396367d-7a1d-4a1b-a320-91bd25f1e9cb%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3d7428c8-2245-4ba1-9109-46c8877f1913%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3d7428c8-2245-4ba1-9109-46c8877f1913%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:32am UTC](https://discuss.elastic.co/t/es-1-4-2-snapshot-fs-repository-directory-structure/22181/4 "2017-07-06T00:32:31Z")