# How to refresh/update snapshot repositories?

**URL:** https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572
**Category:** Elasticsearch
**Created:** [February 9, 2017, 9:49pm UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572 "2017-02-09T21:49:20Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![MathSmath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mathsmath/32/15390_2.png) [@MathSmath](https://discuss.elastic.co/u/MathSmath)
#### Post date: [February 9, 2017, 9:49pm UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572/1 "2017-02-09T21:49:20Z")

</div>

I'm moving data from cluster 2x.A to cluster 5x.B, and have the same S3-based repo registered on both. But any _new_ snapshots taken on A don't show up on B.

I've tried updating B's repo registration with both PUT and PATCH, and neither one updates the list of snapshots. I've even unregistered the repo on B, made sure it was no longer listed as a repo, and reregistered it. But I still only see snapshots from the time I originally registered the repo on B.

The new snapshot files and related index data are definitely on S3. And the new snapshots are visible from cluster A.

Does each cluster maintain an independent cache of snapshots, that only gets updated when certain actions are taken? If so, is there a way to have a cluster refresh this cache?

---

<div class="post-metadata">

### Author: ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)
#### Post date: [February 10, 2017, 1:12am UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572/2 "2017-02-10T01:12:12Z")

</div>

I don't think you can/should write with a 2.x cluster on a repository which has been upgraded by a 5.x cluster.

One thing you can try is to delete the repo in 5.x and create it again but I'm not sure it will work.

---

<div class="post-metadata">

### Author: ![MathSmath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mathsmath/32/15390_2.png) [@MathSmath](https://discuss.elastic.co/u/MathSmath)
#### Post date: [February 10, 2017, 3:56pm UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572/3 "2017-02-10T15:56:09Z")

</div>

Huh. The docs say: "A snapshot of an index created in 2.x can be restored to 5.x.". Are you saying that the 5x cluster has updated the _repo itself_ somehow?

I did unregister the repo in 5x (with the DELETE verb), and register it again. And it still doesn't show the new 2x snapshots. Even tried giving the repo a different name.

---

<div class="post-metadata">

### Author: ![abeyad](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@abeyad](https://discuss.elastic.co/u/abeyad)
#### Post date: [February 10, 2017, 4:37pm UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572/4 "2017-02-10T16:37:35Z")

</div>

A snapshot taken in 2.x can indeed be restored in 5.x However, there are two points:

1. You should never have two different clusters connected to the same repository where both clusters can write to the repository. This is dangerous and can lead to multiple race conditions, even if they are both 2.x clusters. All but one of the clusters should open the repository in a `readonly` fashion.

2. In 5.x, the formats also changed to make repositories more resilient. In particular, we introduced the notion of generational `index-N` files, as opposed to 2.x which just had one `index` blob that kept getting overwritten. The `index` blobs maintain the truth of what are the valid snapshots in the repository. When you connect a 5.x cluster to a repository that was created in 2.x, it will notice that the blobs are in the older 2.x format, so it will "upgrade" them to the `index-N` version. Thereafter, only 5.x clusters should write to the repository. A 2.x cluster that writes a snapshot will only update the `index` blob, not the `index-N` blob, and therefore, the 5.x cluster will have no clue about the new snapshots.

If you want to have the 5.x cluster pick up the new snapshots, do the following:

1. Unregister the repository from the 5.x cluster
2. Delete all blobs of the `index-N` format (i.e. `index-1`, `index-2` etc). But do _not_ delete the `index` blob (the one without a "-N" at the end).
3. Delete the `index.latest` blob
4. Set the 2.x cluster's repository to `readonly`
5. Re-register the repository on the 5.x cluster.

Hope this helps clarifying.

---

<div class="post-metadata">

### Author: ![MathSmath](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mathsmath/32/15390_2.png) [@MathSmath](https://discuss.elastic.co/u/MathSmath)
#### Post date: [February 10, 2017, 7:53pm UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572/5 "2017-02-10T19:53:58Z")

</div>

Thank you for the detailed answer. That helps a lot!

This advice and background should be called out in the snapshot and restore docs, IMO.

---

<div class="post-metadata">

### Author: ![abeyad](https://avatars.discourse-cdn.com/v4/letter/a/278dde/32.png) [@abeyad](https://discuss.elastic.co/u/abeyad)
#### Post date: [February 10, 2017, 8:05pm UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572/6 "2017-02-10T20:05:27Z")

</div>

You are correct, I will make a note to update the docs with this info

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 10, 2017, 8:05pm UTC](https://discuss.elastic.co/t/how-to-refresh-update-snapshot-repositories/74572/7 "2017-03-10T20:05:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
