Elasticsearch Incremental Snapshot


(krishna singh) #1

Let me explain with an example:

Snapshot1 pushed segments 1, 2, 3 in repository X
Snapshot2 needs to backup segments 1, 2, 3, 4 (4 contains the new data). But repository X already has segments 1, 2, 3. So the _snapshot action will only copy for snapshot2 the segment 4.

When you delete the snapshot1, as snapshot2 also has links to segments 1, 2, 3 those files are not removed.

When you restore snapshot2, you restore all files linked: segments 1, 2, 3 and 4.

I want only new segment(only 4) will backup....is there any possible way??


(Kartheek Gummaluri) #2

I didn't get you give me brief question so that I can answer


(krishna singh) #3

thnxsss kartheek for reply....I want to take snapshot of only new Index not the all data again.. ..Is it possible??I am trying it take snapshot of whole data again.


(Kartheek Gummaluri) #4

Do you have marvel


(krishna singh) #5

No, I dont have...But I have HQ Plugin.


(Kartheek Gummaluri) #6

ok then try this query which is used to take snapshot for particular indices

curl -XPUT "http://localhost:9200/_snapshot/my_backup/snapshot_2" -d'
{
"indices": "index_2"
}'


(krishna singh) #7

yeah sure this will work fine. but what would be if i have single index with crores of data and everytime i dont want to backup whole index.
I want to snapshot or backup only newly indexed data not the whole previously indexed data again.
Is there any way to do this?


(David Pilato) #8

If you launch a new backup under the same repository, then it will be incremental. But you already know that.

I don't understand what you want more.


(Christian Dahlqvist) #9

I would recommend reading this blog post, which describes how it works quite well.


(system) #10