Hi All,
We're using the snapshot API to backup our ES cluster and as time advances,
the repository storage utilization is growing.
Currently I use 'rsync' to copy snapshot repository data outside of our
production environment, but eventually I'd need to delete some snapshots to
free space for new ones.
The repository keeps files like 'index', 'snapshot-*' and 'metadata-' files
and I suspect that deleting these files will fail the repository from being
functional.
Is there a way that i can 'rsync' the snapshot repository to another
location and delete the existing snapshots?
Can I delete that snapshot data itself (the 'indices' directory the resides
in the repository) without damaging the repository itself?
In these sorts of cases it may make sense to have a repo for a week/months
worth of snapshots, then just retain however much you need in these rolling
repos. Kind of like what LS does for its indices.
Hi All,
We're using the snapshot API to backup our ES cluster and as time
advances, the repository storage utilization is growing.
Currently I use 'rsync' to copy snapshot repository data outside of our
production environment, but eventually I'd need to delete some snapshots to
free space for new ones.
The repository keeps files like 'index', 'snapshot-*' and 'metadata-'
files and I suspect that deleting these files will fail the repository from
being functional.
Is there a way that i can 'rsync' the snapshot repository to another
location and delete the existing snapshots?
Can I delete that snapshot data itself (the 'indices' directory the
resides in the repository) without damaging the repository itself?
If I understand you correctly, the monthly/weekly repository will get
purged when we decide to, right?
Another solution crossed my mind:
Thinking of HDFS, which I'm not so familiar with, but is there some kind of
HDFS node which is similar to ES node which is not a master nor a data?
This way all snapshot will go through a HDFS 'gateway' node to outside of
our prod env.
On Monday, February 16, 2015 at 11:30:14 AM UTC+2, Mark Walkom wrote:
In these sorts of cases it may make sense to have a repo for a week/months
worth of snapshots, then just retain however much you need in these rolling
repos. Kind of like what LS does for its indices.
Hi All,
We're using the snapshot API to backup our ES cluster and as time
advances, the repository storage utilization is growing.
Currently I use 'rsync' to copy snapshot repository data outside of our
production environment, but eventually I'd need to delete some snapshots to
free space for new ones.
The repository keeps files like 'index', 'snapshot-*' and 'metadata-'
files and I suspect that deleting these files will fail the repository from
being functional.
Is there a way that i can 'rsync' the snapshot repository to another
location and delete the existing snapshots?
Can I delete that snapshot data itself (the 'indices' directory the
resides in the repository) without damaging the repository itself?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.