I've checked on here for anything similar but didn't find anything. Hopefully you have some suggestions. And before I start I'm a Linux admin so that's how I'm thinking in my post .
- Standard ELK setup with syslog --> logstash (parsed) and stored in ES.
- I keep 2 weeks of indexes open and searchable. The rest are left in my Directory and flushed/closed.
- Daily index sizes are around 8GB.
As you can image this is starting to take up quite a bit of space so I want to move them off to an archive area for when they may be required in the future. I've already tested tar'ing up and gzip'ing a closed index and deleting the original directory, then unzip'ing and re-opening the index in ES and that works.
It's clean and quick but maybe not supported.
I've looked at archiving and backup but I have some questions:
- it doesn't work on closed indexes (i read something about partial: true?).
- What happens after you run the archiving? is the original index removed from the original location while the archive exists in your archive location? The aim is to move the indexes out of their current location to save space.
- I'm not sure this is what I'm looking for.
- I'm happy to provide more information.
I've probably got a few things wrong but I look forward to your comments.