Snapshot backup and recovery for elasticseach cluster


(Rahul Sharma) #1

Hi All,

We are looking to have two production environments(one primary and other secondary for disaster), and want to snapshot entire cluster and replay it on a secondary one (secondary is for disaster switch) from time to time.
Can you please tell what is the best way to take snap shot from primary and replay the snapshot in secondary?

How often we can take snap shot if data hosted on ES is around 50TB?

How much time it takes to replay the snap shot on secondary cluster?

Where should we store snap shots for faster replay on secondary? (there are few options s3, hdfs etc).
We only have access to magnetic disks.

Thanks a lot in advance for your help.

Regards,
Rahul


(Mark Walkom) #2

Use the Snapshot and Restore functionality, then copy it over somehow and restore.

Snapshots are deltas, so as often as you want.

That's too hard for us to answer, how fast is your network/disk/CPU, how big is the snapshot?

That's, again, hard to say as you need to pay some cost of shipping the data from one site to another, so what's the lowest cost of that in terms of time/bandwidth?

It might also be worth looking into some of the options discussed here - https://www.elastic.co/blog/scaling_elasticsearch_across_data_centers_with_kafka


(system) #3