Can we take a snapshot of a particular shard of an index in ES ?
No you can't but I'm curious about your use case...
I want to take snapshots of individual nodes of the Elasticsearch so that if any of the node fails, i can restore that particular snapshot onto some new ES node and reclaim any lost data.
I do not have replication enabled since i am using SSDs and replication is costly to the project.
So, I am planning to take snapshots of each of the nodes onto hard-drives and restore as needed.
Apart having one single shard per index, I don't see how you can do it.
What kind of data are you indexing? Logs? Time based data?
Thanks @dadoonet for taking interest.
I am indexing tweet data into monthly indexes with shards spread across the cluster(instead of a single shard index).
Apart from replication, what do you suggest for high availability of ES data ?
We are currently planning to take backup of the cluster onto a backup machine with RAID disks.
Backing-up a single node would ease the restore process. If any of the data node fails, just take the restore from the snapshot of that particular node.
However, if each of the individual indexes are backed-up, we'll have to run backup command for each of the index which resided on the failed node.
Any ideas on this ?
Thanks