Is the snapshot incremental?

IronMike · August 6, 2014, 4:36pm

curl -XPUT http://localhost:9200/snapshot/myRepository/myIndexdate "+%Y-%m-%d"?wait_for_completion=true

This cron job runs daily which backs up my index to AWS S3, each day the
snapshot has a different name.

I want to make sure that I am not duplicating a 10GB index for example
everyday in S3? Does it look at my index from yesterday and only index the
changes? What if there were no changes, What does it mean for todays
snapshot vs yesterday's snapshot (Is there a duplicate?)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/28ea0ac8-03c2-44f4-82ca-00ac288b45e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dadoonet · August 6, 2014, 5:29pm

Well. It is incremental.

But let's say you have saved old Lucene segments and that old segments has been merged in the meantime to a new bigger one, the next snapshot will copy the new BIG segment and remove the old ones.

It means that old data will be copied twice in this scenario.

Makes sense?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 août 2014 à 18:36, IronMike sabdalla80@gmail.com a écrit :

curl -XPUT http://localhost:9200/_snapshot/myRepository/myIndex_`date "+%Y-%m-%d"`?wait_for_completion=true

This cron job runs daily which backs up my index to AWS S3, each day the snapshot has a different name.

I want to make sure that I am not duplicating a 10GB index for example everyday in S3? Does it look at my index from yesterday and only index the changes? What if there were no changes, What does it mean for todays snapshot vs yesterday's snapshot (Is there a duplicate?)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/28ea0ac8-03c2-44f4-82ca-00ac288b45e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/C085A7A2-984D-4707-8892-B97FD9243AFD%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

IronMike · August 6, 2014, 6:04pm

Thanks, it makes sense in this case. I don't think I can prevent something
like that from happening?

On Wednesday, August 6, 2014 1:29:40 PM UTC-4, David Pilato wrote:

Well. It is incremental.

But let's say you have saved old Lucene segments and that old segments has
been merged in the meantime to a new bigger one, the next snapshot will
copy the new BIG segment and remove the old ones.

It means that old data will be copied twice in this scenario.

Makes sense?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 août 2014 à 18:36, IronMike <sabda...@gmail.com <javascript:>> a
écrit :

curl -XPUT http://localhost:9200/_snapshot/myRepository/myIndex_`date
"+%Y-%m-%d"`?wait_for_completion=true

This cron job runs daily which backs up my index to AWS S3, each day the
snapshot has a different name.

I want to make sure that I am not duplicating a 10GB index for example
everyday in S3? Does it look at my index from yesterday and only index the
changes? What if there were no changes, What does it mean for todays
snapshot vs yesterday's snapshot (Is there a duplicate?)

Thanks

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/28ea0ac8-03c2-44f4-82ca-00ac288b45e6%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/28ea0ac8-03c2-44f4-82ca-00ac288b45e6%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/afc9e207-5dca-47b7-a8d6-147ba0d0423b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

dadoonet · August 6, 2014, 6:36pm

No. I don't think so.

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 août 2014 à 20:04, IronMike sabdalla80@gmail.com a écrit :

Thanks, it makes sense in this case. I don't think I can prevent something like that from happening?

On Wednesday, August 6, 2014 1:29:40 PM UTC-4, David Pilato wrote:
Well. It is incremental.

But let's say you have saved old Lucene segments and that old segments has been merged in the meantime to a new bigger one, the next snapshot will copy the new BIG segment and remove the old ones.

It means that old data will be copied twice in this scenario.

Makes sense?

--
David
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 6 août 2014 à 18:36, IronMike sabda...@gmail.com a écrit :

curl -XPUT http://localhost:9200/_snapshot/myRepository/myIndex_`date "+%Y-%m-%d"`?wait_for_completion=true

This cron job runs daily which backs up my index to AWS S3, each day the snapshot has a different name.

I want to make sure that I am not duplicating a 10GB index for example everyday in S3? Does it look at my index from yesterday and only index the changes? What if there were no changes, What does it mean for todays snapshot vs yesterday's snapshot (Is there a duplicate?)

Thanks

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/28ea0ac8-03c2-44f4-82ca-00ac288b45e6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/afc9e207-5dca-47b7-a8d6-147ba0d0423b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/A2EAD035-26F2-474D-AEB9-822792B55808%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.