Problem with index restoration

Elastic Stack Elasticsearch
1

Hello,
Today, I've noticed a strange behaviour of my elasticsearch server (7.1.0). When I tried to restore an index from a tar archive nothing happened. Later I noticed that my index didn't exist at elasticearch repository because after server restart it was removed. And I don't know why. Moreover, I can find any useful information about what happened with my index after server restart.
When I use elasticsearch 6.6.1 everything works fine.
So where can I find a couse of that situation?

Regards,
Arek

2

Was this index created with a previous version of elasticsearch like 5.x or 2.x?

3

Index was created with 7.1.0. elasticsearch....

4

So I don't understand what you did.

Start a 7.1 new cluster
Create an index
Snapshot it
Delete it
Restore it

Is that what you did?

5

Cluster (in fact standalone server) was created with 6.6.1 elasticsearch version.
All indices I backup by my own script in tar archive (I do not use snapshot feature)
And as I said before, restore procedure worked perfect. Then I upgraded server to 7.1.0 and all indices were properly reindexed to version 7.1.0 or ,since upgrade, created with version 7.1.0.
And, yes, I delete unused indices. But now I have to restore one of deleted ones from my archive and I can't do that because that restored index (its folder) disapears shortly after server restarts.

6

But now I have to restore one of deleted ones from my archive

Which archive? An archive that has been created with 6.6.1 or 7.1.0?

I can't do that because that restored index (its folder) disapears shortly after server restarts.

Doing manual stuff in the data dir is not supported. You should always use the APIs we are providing. Here the snapshot and restore API.

1 Like
7

Which archive? An archive that has been created with 6.6.1 or 7.1.0?

7.1.0

Doing manual stuff in the data dir is not supported. You should always use the APIs we are providing. Here the snapshot and restore API.

OK. I understand, however in previous version such data manipulation works ok.
So, is it possible to restore any folder with data into elasticsearch? If not, I have a huge problem...

8

You can save and restore probably the full data dir. I don't know if you can do it partially. @DavidTurner probably knows.

9

It might have appeared to work ok but I don't know that this has ever really worked, and there's a big warning in the 6.6 docs saying specifically not to do this:

Attempts to restore a cluster from such a backup may fail, reporting corruption and/or missing files, or may appear to have succeeded having silently lost some of its data. The only reliable way to back up a cluster is by using the snapshot and restore functionality.

Although you say that this has changed in 7.1.0, I think you would have seen the same behaviour in 6.6.1 too. It might work to restore it into a new cluster instead, but really we can't guarantee anything about this process. There are no user-serviceable parts inside the data directory.

1 Like
10

I should add that even if you manage to get it to work now, you should expect this kind of process to stop working in some future version, without warning. It may start completely failing, or may work intermittently, or may silently lose some of your data.

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.