Synced Snapshots in S3 don't show up as available in the Repository

Elastic Stack Elasticsearch
1

I'm new to Elasticsearch and trying to copy an index from a Dev environment to a Test environment.

I've installed the S3 Repository plugin in both Dev and Test and configured the repository in Kibana. I setup a policy in the Dev environment to create backups on a nightly basis.

I synced the Dev S3 bucket to the Test S3 repository bucket. The Test S3 repo seems to validate fine in Kibana. When I go into the "Snapshot and Restore" section of Kibana though and try to restore a snapshot to the Test environment, it says that I don't have any snapshots yet.

Is there something that I need to do to make "Snapshot and Restore" aware of the snapshot files that have been copied over?

Thanks in advance!

2

Hi @mattph and welcome.

You're effectively trying to restore the repository in your test environment from a repository backup, but skipping some of the steps described in the manual. Note in particular:

When restoring a repository from a backup, you must not register the repository with Elasticsearch until the repository contents are fully restored. If you alter the contents of a repository while it is registered with Elasticsearch then the repository may become unreadable or may silently lose some of its contents.

1 Like
3

Hi David, thank you for your quick response.

Since the documentation you referenced indicated that the registration can only happen after the files are there, I went ahead and removed the repository in the Test environment and then tried to recreate the repo. That still didn't seem to work, although I received a different result. On the Snapshots tab, I received the message "Some repositories contain errors" and it still did not display the snapshots.

What did work was for me though, was to again remove the Test environment repository and recreate it, this time pointing directly to the Dev S3 Bucket (after setting the appropriate bucket policies). Once I did that, the Test Repo was able to see the snapshots. With that in place, I was able to perform a restore of the index that I was interested in to the Test cluster.

I then dropped the Test repository (since it was pointing to the Dev S3 bucket) and recreated it pointing to the empty Test environment S3 bucket. I then created a policy and performed a manual snapshot of the Test index, which worked fine.

Should copying the Dev repository snapshot files into the Test S3 bucket have worked? It almost seems like the path is embedded somewhere in the repository contents, preventing a copy to a different location?

Thanks again for the help and pointing me in the right direction.

-Matt

4

Yes, as long as it was a completely accurate copy that was either performed atomically or else it was taken when Elasticsearch wasn't writing to the repository.

That's not the case.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.