WorkFlow for Snapshotting Indices as a Backup Option

[ninjada]

Hello, currently trying to debug some issues with an ELK stack, where I've moved the db from an instance to the AWS ElasticSearch instance.. Caused a few issues and now i'm generally just wondering about the workflow i should be approaching with snapshotting indices and any requirements..

I currently have 37 days of indices, i'd like to backup and have ongoing management of automated via curator/curator_cli

I'd like to do two things, have every day snapshotted and any indice older than 28 days deleted.

Is daily snapshotting too much? could i also run a snapshot operation every 14 days for the last fortnights indices and a delete operation every 28 days? or does that effect how they're restored? ie. if you snapshot last 14 days of indices do you have to restore them in blocks of 14 days?

Is this sound for backup purposes? For instance if 365 days go by, and say i need to restore all of July 2018's indices, i'll be okay right?

[akhileshsingh]

Hi ninjada,

Yes, this curator snapshot is for backup purpose and you can restore according to your need with the help of filtertypes available in curator.

Daily snapshot and snapshot operation after 14 days depends on you and you can recover in blocks or either you can recover a particular index.

Create two action file:

1. Snapshot
2. Delete indices
   Use filters according to your usecase in both action file.
   And if you want to execute both action file on every 14 and 28 days then you need to make entry in crontab -e.

Below are the links for your reference to create action file:
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/snapshot.html
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/ex_delete_indices.html

Below is the link for filters:
https://www.elastic.co/guide/en/elasticsearch/client/curator/current/filters.html

I hope the above step will help you
please feel free to reply if you have any query.

Thanks.