# Listing stored snapshots - RepositoryMissingException - New, empty cluster **URL:** \ **Category:** Elasticsearch\ **Created:** [February 21, 2014, 11:22pm UTC](https://discuss.elastic.co/t/listing-stored-snapshots-repositorymissingexception-new-empty-cluster/15962 "2014-02-21T23:22:05Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Tony\_Su](https://avatars.discourse-cdn.com/v4/letter/t/8dc957/32.png) [@Tony\_Su](https://discuss.elastic.co/u/Tony_Su)\ **Post date:** [February 21, 2014, 11:22pm UTC](https://discuss.elastic.co/t/listing-stored-snapshots-repositorymissingexception-new-empty-cluster/15962/1 "2014-02-21T23:22:05Z")
Am testing on a lab cluster 5 nodes. Repository name = "Snapshots" Immediately after creating the snapshots, listing the snapshots returned the snapshots and the indices they contained. Now, am attempting to restore the snapshots to an empty cluster. The new cluster looks good from the following: # curl -XGET localhost:9200/\_cluster/health {"cluster\_name":"elasticsearch","status":"green","timed\_out":false,"number\_of\_nodes":5,"number\_of\_data\_nodes":5,"active\_primary\_shards":1,"active\_shards":2,"relocating\_shards":0,"initializing\_shards":0,"unassigned\_shards":0} From the master node on this new cluster, ran the following to list all snapshots eligible to be restored. The node that's named is the local node. Have verified the node has full access to the shared file system and access to the snapshot repository. # curl -XGET localhost:9200/\_snapshot/Snapshots/\_all {"error":"RemoteTransportException[[ELASTICSEARCH-1][inet[/192.168.248.147:9300]][cluster/snapshot/get]]; nested: RepositoryMissingException[[Snapshots] missing]; ","status":404} Am trying to think of reasons why ELASTICSEARCH-1 can't find the snapshot repository. Maybe no other node can "find" the repository also, but the command stops at this first node's failure. Does the repository need to be registered using the steps to initialize an empty repository before creating snapshots? If so, I assume the existing repository data won't be wiped out (maybe the data should be moved, repository created and then data moved back to the new registered repository?) Maybe Snapshot/Restore has not actually been trialed in a migration scenario? Note this lab exercise has similarities to another Forum post asking about Cluster migration and his proposal to use Snapshot/Restore also. Thx, Tony -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/65f1f916-e413-4b59-abdd-55e98f64681f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/65f1f916-e413-4b59-abdd-55e98f64681f%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Binh\_Ly](https://avatars.discourse-cdn.com/v4/letter/b/ce7236/32.png) [@Binh\_Ly](https://discuss.elastic.co/u/Binh_Ly)\ **Post date:** [February 21, 2014, 11:48pm UTC](https://discuss.elastic.co/t/listing-stored-snapshots-repositorymissingexception-new-empty-cluster/15962/2 "2014-02-21T23:48:51Z")
When you originally defined the snapshot, you did something like this (or similar): PUT [http://localhost:9200/\_snapshot/Snapshots](http://localhost:9200/_snapshot/Snapshots) { "type": "fs", "settings": { "location": "/blah" } } On the new/empty cluster, this snapshot is not yet registered, so the first thing you need to do is to run that again: PUT [http://localhost:9200/\_snapshot/Snapshots](http://localhost:9200/_snapshot/Snapshots) { "type": "fs", "settings": { "location": "/blah" } } Then after that, you should be able to restore to the new/empty cluster (as long as that location "/blah" is accessible from the new cluster node). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/65f4f381-ec15-4083-9608-9ca02dfc10a8%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/65f4f381-ec15-4083-9608-9ca02dfc10a8%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Tony\_Su](https://avatars.discourse-cdn.com/v4/letter/t/8dc957/32.png) [@Tony\_Su](https://discuss.elastic.co/u/Tony_Su)\ **Post date:** [February 22, 2014, 12:56am UTC](https://discuss.elastic.co/t/listing-stored-snapshots-repositorymissingexception-new-empty-cluster/15962/3 "2014-02-22T00:56:13Z")
Cool. Still not knowing whether the act of registering a snapshot repository pointing to a populated directory might "initialize/wipe" the location, I made a copy before registering again. After registering the repository, I queried for existing snapshots and all looks well, so it looks like registering is non-destructive. Tony On Friday, February 21, 2014 3:48:51 PM UTC-8, Binh Ly wrote: > When you originally defined the snapshot, you did something like this (or > similar): > > PUT [http://localhost:9200/\_snapshot/Snapshots](http://localhost:9200/_snapshot/Snapshots) > { > "type": "fs", > "settings": { > "location": "/blah" > } > } > > On the new/empty cluster, this snapshot is not yet registered, so the > first thing you need to do is to run that again: > > PUT [http://localhost:9200/\_snapshot/Snapshots](http://localhost:9200/_snapshot/Snapshots) > { > "type": "fs", > "settings": { > "location": "/blah" > } > } > > Then after that, you should be able to restore to the new/empty cluster > (as long as that location "/blah" is accessible from the new cluster node). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6d7dc569-7c04-4068-af81-9f59935ebf8d%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6d7dc569-7c04-4068-af81-9f59935ebf8d%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Tony\_Su](https://avatars.discourse-cdn.com/v4/letter/t/8dc957/32.png) [@Tony\_Su](https://discuss.elastic.co/u/Tony_Su)\ **Post date:** [February 22, 2014, 1:56am UTC](https://discuss.elastic.co/t/listing-stored-snapshots-repositorymissingexception-new-empty-cluster/15962/4 "2014-02-22T01:56:15Z")
Wow! Am amazed how performant a restore is compared to restarting a populated cluster! Existing cluster: 5 nodes, all master and data eligible Configured, 5shards,0replica/index Apache log data 330 indices 4361031 documents current configuration to prevent cluster from considering re-allocation until all nodes is the following (set to same number as total nodes in cluster) gateway.recover\_after\_nodes: 5 When restarting this cluster, I usually have trouble getting all nodes to stay online during restart. This typically causes shard re-allocation, followed possibly by more re-allocation. Typically, minimally it takes 10 min for the cluster to recognize all 5 nodes Followed by minimally 40 minutes (often more) to go green. When restoring this cluster from snapshot Since there is no data in the new cluster, 5 nodes are recognized almost immmediately (no 10 min wait) Restore (not completely successful, see below) is only about 20 minutes. Not even a hint that any node was unhappy and might drop from the cluster. Noticed - Shard initialization is 20 shards at a time, twice as many as when restarting a populated cluster. But, looks like two shards remain unallocated - A replica of a Marvel index (the Primary appears to have successfully initialized) - A primary of one of the Apache log indices. Last time I experienced this, I simply deleted the problem index and re-input the data (BTW - Input is by month, so to restore the one day involves input 30 days. For the other 29 days, the version was incremented which better answers a post to this Forum weeks ago when I asked about this scenario) In this scenario, I don't think I can re-load historical Marvel data (does it even exist in a way that can be re-loaded?) Also, am wondering if there is a better solution to "fixing" a single index that refuses to initialize. TIA, Tony On Friday, February 21, 2014 4:56:13 PM UTC-8, Tony Su wrote: > Cool. > > Still not knowing whether the act of registering a snapshot repository > pointing to a populated directory might "initialize/wipe" the location, I > made a copy before registering again. > > After registering the repository, I queried for existing snapshots and all > looks well, so it looks like registering is non-destructive. > > Tony > > On Friday, February 21, 2014 3:48:51 PM UTC-8, Binh Ly wrote: > > > When you originally defined the snapshot, you did something like this (or > > similar): > > > > PUT [http://localhost:9200/\_snapshot/Snapshots](http://localhost:9200/_snapshot/Snapshots) > > { > > "type": "fs", > > "settings": { > > "location": "/blah" > > } > > } > > > > On the new/empty cluster, this snapshot is not yet registered, so the > > first thing you need to do is to run that again: > > > > PUT [http://localhost:9200/\_snapshot/Snapshots](http://localhost:9200/_snapshot/Snapshots) > > { > > "type": "fs", > > "settings": { > > "location": "/blah" > > } > > } > > > > Then after that, you should be able to restore to the new/empty cluster > > (as long as that location "/blah" is accessible from the new cluster node). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/b1cd7695-bfce-4d57-bdc7-bac09b039071%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/b1cd7695-bfce-4d57-bdc7-bac09b039071%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Tony\_Su](https://avatars.discourse-cdn.com/v4/letter/t/8dc957/32.png) [@Tony\_Su](https://discuss.elastic.co/u/Tony_Su)\ **Post date:** [February 22, 2014, 8:33am UTC](https://discuss.elastic.co/t/listing-stored-snapshots-repositorymissingexception-new-empty-cluster/15962/5 "2014-02-22T08:33:47Z")
> Update: After shutting down the cluster and restarting, the problem resolved itself. Both shards which refused to initialize earlier were able to initialize on restart. Maybe the lesson to be learned is that assuming that when the snapshot were made, integrity was verified(I hope so!) so restore should almost certainly be good. Tony -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e3d1bef5-04be-4a7c-8a3a-3557c1dd324f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e3d1bef5-04be-4a7c-8a3a-3557c1dd324f%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:48am UTC](https://discuss.elastic.co/t/listing-stored-snapshots-repositorymissingexception-new-empty-cluster/15962/6 "2017-07-06T01:48:08Z")