Snapshot and restore source only feature

Elastic Stack Elasticsearch
1

Hi all
i recently trying the snapshot feature in elastic and i noticed the feature source only button there, according to what i have read the source only will only save the metadata and the field of the index and nothing else. I dont really understand what is the point of having that feature in the snapshot when they will only save really nothing that we need.

I have try reindexing like the doc told me and they do in fact get the data back but i do have to ask where is that data comming from since none of the data is saved during the snapshot.

2

That would be the mappings, which are important.

It's not clear what you mean by this. Which doc are you referring to?

3

i dont really remember where i read it but i once read that after i restore the snapshot i need to reindex, they dont really explain why.

4

Maybe if it's from an older version of Elasticsearch, yes.

5

So can you please spend some time to explain to me how the source only snapshot work and how the data from the source only snapshot when reindex can get back all the data when that feature only backup the metadata and the mapping.

6

It cannot, that is why I mentioned it's not clear.

How do you know it's a source only snapshot? When you restore it, what does the index look like, is there docs in there?

7

i know that because i set the snapshot repository to source only.
Then when i restore the snapshot the index is empty. there is nothing inside of it. Normally even when i have an index that doesnot have any doc they still display the number of doc is 0 and the size of the indices is 500KB for example. But the index that got restored doesnot even have those information.

I thinks i read the doc on the elastic guide page.

8

Yep, an empty index does have a size.

So are you then running a reindex from that restored, empty index into a new one?
If so what does the reindex request look like? Can you show the output of _cat/indices?v for the restored index?

9

I have viewed the indices and it is just like all other indices. The reindexing process is also the same. And since i have already delete the original indices that before the snapshot so i cant really compare the number of doc and the size.

10

Without some more information, as asked for above, it's really hard to identify what is happening.

But ultimately, you cannot reindex an empty index.

11

So really what is really the point of the source only snapshot then, since i can only backup the mapping. And like what you said we cant reindex an empty index.

Also i will do some more testing with this and try to identify the problems.

12

Ah ok, so you are talking about source only snapshots. That explains things then.

A source repository enables you to create minimal, source-only snapshots that take up to 50% less space on disk. Source only snapshots contain stored fields and index metadata. They do not include index or doc values structures and are not searchable when restored. After restoring a source-only snapshot, you must reindex the data into a new index.

I bolded the appropriate part that explains how that process works.

13

Yes that is the part that i read from the doc. So according to that text if i reindex those indices then i can get the data back right.

14

Yes.

15

So if i have an index that have 100 doc in them, take the snapshot data that i got then move them to another cluster, then i restore them and reindex them. Will i get back full 100 doc with all the normal function like search and aggregate ability from elastic with full information?

16

It would depend on your actual reindex, but that's the idea, yes.

17

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.