ES snapshot of a specific shard

(Ashish Belokar) #1

Can we take a snapshot of a particular shard of an index in ES ?

(David Pilato) #2

No you can't but I'm curious about your use case...

(Ashish Belokar) #3

I want to take snapshots of individual nodes of the Elasticsearch so that if any of the node fails, i can restore that particular snapshot onto some new ES node and reclaim any lost data.
I do not have replication enabled since i am using SSDs and replication is costly to the project.
So, I am planning to take snapshots of each of the nodes onto hard-drives and restore as needed.

(David Pilato) #4

Apart having one single shard per index, I don't see how you can do it.

What kind of data are you indexing? Logs? Time based data?

(Ashish Belokar) #5

Thanks @dadoonet for taking interest.
I am indexing tweet data into monthly indexes with shards spread across the cluster(instead of a single shard index).
Apart from replication, what do you suggest for high availability of ES data ?
We are currently planning to take backup of the cluster onto a backup machine with RAID disks.
Backing-up a single node would ease the restore process. If any of the data node fails, just take the restore from the snapshot of that particular node.
However, if each of the individual indexes are backed-up, we'll have to run backup command for each of the index which resided on the failed node.
Any ideas on this ?

(system) #6