# Snapshot files are created every time even no change in index **URL:** \ **Category:** Elasticsearch\ **Created:** [May 28, 2015, 11:13am UTC](https://discuss.elastic.co/t/snapshot-files-are-created-every-time-even-no-change-in-index/23707 "2015-05-28T11:13:34Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![shoebalig](https://avatars.discourse-cdn.com/v4/letter/s/c57346/32.png) [@shoebalig](https://discuss.elastic.co/u/shoebalig)\ **Post date:** [May 28, 2015, 11:13am UTC](https://discuss.elastic.co/t/snapshot-files-are-created-every-time-even-no-change-in-index/23707/1 "2015-05-28T11:13:34Z")
I have a cron-job runs daily. There are chances that there is no modification or changes found in original index. however cron-job create snapshot file again on its scheduled time. Due to which number of snapshot files is too large. which leads restoration of index, takes a lot of time. Is there anyway I should create snapshot file only when there is a difference in index data? What if I retain only last 5 or last 10 snapshot? Will this cause any data lose anyway? And on restore whole index will be restored? Please suggest. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 28, 2015, 10:07pm UTC](https://discuss.elastic.co/t/snapshot-files-are-created-every-time-even-no-change-in-index/23707/2 "2015-05-28T22:07:06Z")
Snapshots are taken on the shard level, so if a shard changes due to things like merging, then it needs to record this. You can delete snapshots, ES will just hold onto the shards it needs. PS - We're moving to [https://discuss.elastic.co/](https://discuss.elastic.co/), please join us there for any future discussions! On 28 May 2015 at 21:13, shoebalig [shoebalig@gmail.com](mailto:shoebalig@gmail.com) wrote: > I have a cron-job runs daily. There are chances that there is no > modification or changes found in original index. however cron-job create > snapshot file again on its scheduled time. Due to which number of snapshot > files is too large. which leads restoration of index, takes a lot of time. > Is there anyway I should create snapshot file only when there is a > difference in index data? > What if I retain only last 5 or last 10 snapshot? Will this cause any data > lose anyway? And on restore whole index will be restored? > > Please suggest. > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Snapshot-files-are-created-every-time-even-no-change-in-index-tp4074965.html](http://elasticsearch-users.115913.n3.nabble.com/Snapshot-files-are-created-every-time-even-no-change-in-index-tp4074965.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1432811614768-4074965.post%40n3.nabble.com](https://groups.google.com/d/msgid/elasticsearch/1432811614768-4074965.post%40n3.nabble.com) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9kFwpxBaDZt0C\_HzUG\_U5zHpTWgGBqfReEzrB0WK1umg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9kFwpxBaDZt0C_HzUG_U5zHpTWgGBqfReEzrB0WK1umg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Allan\_Mitchell](https://avatars.discourse-cdn.com/v4/letter/a/edb3f5/32.png) [@Allan\_Mitchell](https://discuss.elastic.co/u/Allan_Mitchell)\ **Post date:** [May 29, 2015, 8:17am UTC](https://discuss.elastic.co/t/snapshot-files-are-created-every-time-even-no-change-in-index/23707/3 "2015-05-29T08:17:38Z")
Mark This seems counter intuitive. Incremental to me and technologies with which I have worked before means that in order to restore to a known state you need A+B+C..... In temporal order. Consider this example DELETE /test POST /test { "settings" : { "number\_of\_shards" : 1 }, "mappings" : { "mytype" : { "properties" : { "myfield" : { "type" : "string","analyzer": "standard" } } } } } POST /test/mytype/1 { "myfield": "Doc 1" } Snapshot to snapshot1 POST /test/mytype/2 { "myfield": "Doc 2" } Snapshot to snapshot2 POST /test/mytype/3 { "myfield": "Doc 3" } Snapshot to snapshot3 Delete the test index in ES Now go ahead and delete snapshot2 Restore the snapshot3 backup POST /\_snapshot/FILE\_SYSTEM\_ES/snapshot3/\_restore All 3 documents are in the index Delete the index in ES again Delete snapshot1 delete the index in ES Restore the snapshot3 backup All 3 documents are in the index Even if the snapshot only takes the incremental changes it would seem the previous snapshot is merged with the latest allowing the deletions you see above. To get to my latest position I only need the latest snapshot. If I want to get back to a previous state then I only need the relevant snapshot. I dare say this is just nomenclature but incremental to me does not do what the incremental in ES portrays SQL Server Example. If I take a FULL backup on Sunday and Incrementals on each night of the week. If I delete everything before last night and try to restore only the latest incremental I would be stuck. If I delete snapshots in reverse order then the document count in the index decreases as expected. I cannot find any documentation anywhere on this process. Allan On 28 May 2015, at 23:07, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: Snapshots are taken on the shard level, so if a shard changes due to things like merging, then it needs to record this. You can delete snapshots, ES will just hold onto the shards it needs. PS - We're moving to [https://discuss.elastic.co/](https://discuss.elastic.co/), please join us there for any future discussions! On 28 May 2015 at 21:13, shoebalig [shoebalig@gmail.com](mailto:shoebalig@gmail.com) wrote: > I have a cron-job runs daily. There are chances that there is no > modification or changes found in original index. however cron-job create > snapshot file again on its scheduled time. Due to which number of snapshot > files is too large. which leads restoration of index, takes a lot of time. > Is there anyway I should create snapshot file only when there is a > difference in index data? > What if I retain only last 5 or last 10 snapshot? Will this cause any data > lose anyway? And on restore whole index will be restored? > > Please suggest. > > -- > View this message in context: > [http://elasticsearch-users.115913.n3.nabble.com/Snapshot-files-are-created-every-time-even-no-change-in-index-tp4074965.html](http://elasticsearch-users.115913.n3.nabble.com/Snapshot-files-are-created-every-time-even-no-change-in-index-tp4074965.html) > Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com). > > ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) > > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/1432811614768-4074965.post%40n3.nabble.com](https://groups.google.com/d/msgid/elasticsearch/1432811614768-4074965.post%40n3.nabble.com) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9kFwpxBaDZt0C\_HzUG\_U5zHpTWgGBqfReEzrB0WK1umg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9kFwpxBaDZt0C_HzUG_U5zHpTWgGBqfReEzrB0WK1umg%40mail.gmail.com) [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9kFwpxBaDZt0C\_HzUG\_U5zHpTWgGBqfReEzrB0WK1umg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9kFwpxBaDZt0C_HzUG_U5zHpTWgGBqfReEzrB0WK1umg%40mail.gmail.com?utm_medium=email&utm_source=footer) . For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ## -- Please update your bookmarks! We have moved to [https://discuss.elastic.co/](https://discuss.elastic.co/) You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAECdJzBpOYsTwB9F6w6nNWRp\_nCHmQfZy0fbeFy484Q9969Zog%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAECdJzBpOYsTwB9F6w6nNWRp_nCHmQfZy0fbeFy484Q9969Zog%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:10am UTC](https://discuss.elastic.co/t/snapshot-files-are-created-every-time-even-no-change-in-index/23707/4 "2017-07-06T00:10:53Z")