Are the snapshots made by elasticsearch surely not to have any errors?
Is there any additional option to quickly check the correctness of generated snapshots?
We verify the checksum of every file uploaded to the repository, and verify the checksums again on restore. We can't guarantee that your repository works perfectly of course, that's on you, but at least we will report any errors that it introduces.
Not really, you're expected to take responsibility for the integrity of the data sitting in your repository. If in doubt, use one of the big cloud providers (AWS S3, Google Cloud Storage etc) since they're very reliable.
2 Likes
We verify the checksum of every file uploaded to the repository
Can I somehow "manually" check the checksum of a file uploaded? (correctness)
I have a snapshot made for some resources.
I can make a checksum for snapshot files.
How can I calculate the checksum for indexes/shards in Elasticsearch?
Not directly, no, but you can get Elasticsearch to check all the checksums in a snapshot at once by restoring it into an empty cluster.
So I understand that Elasticsearch itself, automatically calculates checksums of all snapshot files after snapshot execution (in "Taking snapshot")? If the sums are not correct then Elasticsearch will return the error in snapshot status and I will see it in Kibana (snapshot section)?
If the sums are not correct then Elasticsearch will return the error in snapshot status and I will see it in Kibana (snapshot section)?
Yes that's correct.
In addition, it will also mark that copy of the shard as corrupt so that it is not used for any future searches or indexing. If there's a replica then it's promoted to primary and used to rebuild the failed copy; if there's no replica then your cluster health goes red.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.