Queries regarding snapshot APIs

[pinakinab (Pinakin)]

We are trying to use backup/restore feature of Elasticsearch using Azure Storage. We are using OSS version of ES 7.2.0. During the process we came across following queries for which we could not find documentation:

1. Is there any API which gives information about restore process .eg. when was restore started, how long did it take to restore the backup, current status of restore etc. For backup there is an API for these things. It would really help if we could get similar information for restore as well.
2. Once the backup is taken can we check how much is the size of the backup? Current backup status API does not provide this information.
3. Also it would be good to know amount of storage left, amount of storage used, amount of storage total through elasticsearch API.

[Armin_Braun]

Hi @pinakinab

@pinakinab

Is there any API which gives information about restore process .eg. when was restore started, how long did it take to restore the backup, current status of restore etc. For backup there is an API for these things. It would really help if we could get similar information for restore as well.

We don't have an API that tracks the restore as a whole I'm afraid. What we do have though is the cat recoveries api which allows you to get the status and time it took to restore fore each shard in a restore.

@pinakinab

Once the backup is taken can we check how much is the size of the backup? Current backup status API does not provide this information.

There is some nuance to the size of an individual snapshot. Note that each snapshot is incrementally taken on top of all the other snapshots already existing in the repository. So the size of an individual snapshot does not say much. It could be sharing TBs of data with the previous snapshot but itself not have added much data to the snapshot because the cluster's contents didn't change much between snapshots.

That said, you can get the number of files and bytes a snapshot added to the repository via the snapshot shard status API (though again this is on a per-shard basis and you'd need to aggregate the numbers yourself to get it on a per snapshot basis).
This is the API at GET /_snapshot/my_backup/snapshot_1/_status as documented here.

@pinakinab

Also it would be good to know amount of storage left, amount of storage used, amount of storage total through elasticsearch API.

What do you mean here, the snapshot repository or the cluster nodes? If you're backing up to Azure blob storage then the amount of storage left is technically unlimited?
If you're interested in getting the amount of disk space left on each cluster node, you can for example call the cat nodes API that will give you this information for each node.