# Lost es snapshots **URL:** \ **Category:** Elasticsearch\ **Created:** [October 7, 2015, 12:15am UTC](https://discuss.elastic.co/t/lost-es-snapshots/31741 "2015-10-07T00:15:42Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![bluethundr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bluethundr/32/409_2.png) [@bluethundr](https://discuss.elastic.co/u/bluethundr)\ **Post date:** [October 7, 2015, 12:15am UTC](https://discuss.elastic.co/t/lost-es-snapshots/31741/1 "2015-10-07T00:15:42Z")
Hey guys, I'm running elasticsearch 1.7.1. And I'm taking snapshots of the cluster using this cron job: ``` 0 8 * * Sat /bin/curl --user admin:$ES_PASS -XPUT "http://logs.example.com:9200/_snapshot/jf_backup/snapshot_$(date +%Y-%m-%d) ``` Yet, when I try to list my backups with the following command, I get a curious result: ``` [root@logs:/etc/elasticsearch] #curl -XGET 'http://logs.example.com:9200/_snapshot' {} ``` It appears that backups are in fact happening merely because the mount point I established for the backups is filling up: ``` [root@logs:/etc/elasticsearch] #grep path.repo elasticsearch.yml path.repo: ["/mnt/backup", "/mnt/backup/jf_backup"] [root@logs:~] #du -sh /mnt/backup/jf_backup 2.8G /mnt/backup/jf_backup ``` So I'm just wondering what I'm getting wrong here? Any advice on this problem would be welcomed! Thanks ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 7, 2015, 12:42am UTC](https://discuss.elastic.co/t/lost-es-snapshots/31741/2 "2015-10-07T00:42:33Z")
Does it mention anything in the logs? Does `/_snapshot/_all` show anything else? Can you verify the repo? ---
**Author:** ![bluethundr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bluethundr/32/409_2.png) [@bluethundr](https://discuss.elastic.co/u/bluethundr)\ **Post date:** [October 7, 2015, 2:15am UTC](https://discuss.elastic.co/t/lost-es-snapshots/31741/3 "2015-10-07T02:15:34Z")
> [@warkolm](#): > > Does it mention anything in the logs?Does /\_snapshot/\_all show anything else? Can you verify the repo? Warkolm, Nope!! No action in the logs at all when I hit the server with that snapshot command. I was tailing the logs, and put some distance at the end by hitting return a few times. Then issued the command again. Nothing happened in the logs! And /\_snapshot/\_all shows the same thing as my OP ``` [root@logs:~] #curl -XGET 'http://localhost:9200/_snapshot/_all' {} ``` Ran that command from the first ES server itself. And if I try and verify the backup I get a 404: ``` #curl -XPOST 'http://localhost:9200/_snapshot/jf_backup/_verify?pretty=true' { "error" : "RepositoryMissingException[[jf_backup] missing]", "status" : 404 } ``` But if that's truly the case, and there are no backups, then why would the backup directory have data in it? ``` [root@logs:/etc/elasticsearch] #grep path.repo elasticsearch.yml path.repo: ["/mnt/backup", "/mnt/backup/jf_backup"] [root@logs:/etc/elasticsearch] #du -sh /mnt/backup/jf_backup/ 2.8G /mnt/backup/jf_backup/ ``` And I can see this in the backup directories: ``` [root@logs:/etc/elasticsearch] #ls -lh /mnt/backup/jf_backup/ total 12K -rw-r--r--. 1 elasticsearch elasticsearch 0 Aug 23 00:27 index drwxr-xr-x. 55 elasticsearch elasticsearch 4.0K Aug 23 00:23 indices -rw-r--r--. 1 elasticsearch elasticsearch 5.1K Aug 23 00:23 metadata-snapshot_1 -rw-r--r--. 1 elasticsearch elasticsearch 0 Aug 23 08:32 metadata-snapshot_2 -rw-r--r--. 1 elasticsearch elasticsearch 0 Aug 23 00:27 snapshot-snapshot_1 -rw-r--r--. 1 elasticsearch elasticsearch 0 Aug 23 08:32 snapshot-snapshot_2 ``` I'm not certain why that data's there at all if ES seems to think that there are no backups to be found? Odd stuff! Thanks, Tim ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [October 7, 2015, 2:57am UTC](https://discuss.elastic.co/t/lost-es-snapshots/31741/4 "2015-10-07T02:57:29Z")
Those are from August though, perhaps someone setup a repo previously, ran some backups and then removed the repo? ---
**Author:** ![bluethundr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bluethundr/32/409_2.png) [@bluethundr](https://discuss.elastic.co/u/bluethundr)\ **Post date:** [October 7, 2015, 4:33am UTC](https://discuss.elastic.co/t/lost-es-snapshots/31741/5 "2015-10-07T04:33:52Z")
> [@warkolm](#): > > Those are from August though, perhaps someone setup a repo previously, ran some backups and then removed the repo? Quite possibly. But it appears that the backups had gotten a little messed up! So I decided to scrap what was there and just start again. I just rm'd everything in the backup direcotory and bounced all nodes in the cluster. Then re-created the repo. ``` curl --user admin:$ES_PASS -XPOST "http://logs.example.com:9200/_snapshot/jf_backup" -d' { "type": "fs", "settings": { "location": "/mnt/backup/jf_backup/" } }' ``` Set some throttling options: ``` curl --user admin:secret -XPOST "http://logs.example.com:9200/_snapshot/jf_backup" -d' { "type": "fs", "settings": { "location": "/mnt/backup/jf_backup", "max_snapshot_bytes_per_sec" : "50mb", "max_restore_bytes_per_sec" : "50mb" } }' ``` Then listed the backup repo: ``` curl -XGET 'http://logs.example.com:9200/_snapshot/_all?pretty=true' { "jf_backup" : { "type" : "fs", "settings" : { "location" : "/mnt/backup/jf_backup", "max_restore_bytes_per_sec" : "50mb", "max_snapshot_bytes_per_sec" : "50mb" } } } ``` Took a new backup: ``` curl --user admin:$ES_PASS -XPUT "http://logs.example.com:9200/_snapshot/jf_backup/snapshot_$(date +%Y-%m-%d)" {"accepted":true} ``` And was able to verify that the backup had happened: ``` #curl -XPOST 'http://logs.example.com:9200/_snapshot/jf_backup/_verify?pretty=true' { "nodes" : { "95AQx2aiRp64m7S5VlLa4g" : { "name" : "JF_ES3" }, "QS4xFC8DRiSVo9wZE3wTZA" : { "name" : "JF_ES1" }, "VjxijoX4S1ySOkKPPSYztw" : { "name" : "JF_ES2" } } }' ``` Thanks ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:46pm UTC](https://discuss.elastic.co/t/lost-es-snapshots/31741/6 "2017-07-05T23:46:17Z")