ES 6.0 snapshot index UUIDs

Elastic Stack Elasticsearch
1

Hi All,
In ES 2.x, on creation of a snapshot, the directories created in the repository under the 'indices' directory contained the indices names as it is. Given a snapshot name, it was very easy to get all the files (meta-, snap- and indices) for that snapshot and create a single bundled tar file which can be transferred to an offsite location to copy data. Thus we were able to copy data from one cluster to another unconnected cluster.

But, on upgrading to ES 6.1.2, the directory structure created per snapshot under the repository has changed. Although the meta- and snap- files use the UUID of the snapshot, the files under the indices directory seem to be using a temporary UUID which is NOT exposed in any existing APIs.

Is there anyway to map the files under indices directory to its corresponding snapshot? or any API which exposes the UUIDs/IDs used for these indices in the snapshot?

Regards,
Arun

2

Technically the answer is yes, because Elasticsearch does exactly this mapping when restoring from a snapshot, but it varies from version to version and can't be relied upon.

The only reliable way to do this is to tar up the whole repository. You can create a separate repository for this operation if you do not want to transfer all the indices. You can also probably perform an incremental copy of an active repository using something like rsync, as long as you are careful not to attempt to access the destination repository while the copy is in progress.

3

Thanks for reply David.

We are interested in tarballing indices files and the meta&snap files for a particular snapshot and hence the query. This is helpful in the field to move around only the particular data we are interested in.

Assuming ES version 6.1.2, is there an API (or any other method) from which I can get this mapping of indices UUIDs to archived indices for a given snapshot?

-Arun

4

There is no API for this. Perhaps the source helps?

5

Hi David,
Thanks for the pointer.

I could find that index-N files under the repository have data in json format which has the mapping of indices id to their names

Thanks for your help.

-Arun

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.