Restoring a snapshot from one machine to a different machine

Elastic Stack Elasticsearch
1

I would like to use the Snapshot functionality to move indices from an OLD
server to a NEW server. My approach to do this task was to take a snapshot
from OLD server, copy the snapshot over to the NEW server (I zipped this
snapshot because it contained multiple indices), and try to restore that
snapshot from NEW server. However, I am not able to restore my snapshot on
NEW server.

Here are the commands that I've typed.

OLD$ curl -X PUT 'http://localhost:9200/_snapshot/my_backup' -d '{
"type" : "fs",
"settings" : {
"location" : "/mnt1/elasticsearch_backup/my_backup",
"compress" : true
}
}'

OLD$ curl -XPUT
"localhost:9200/_snapshot/my_backup/snapshot_1?wait_for_completion=true"
OLD$ tar cvfz elasticsearch_backup.tgz /mnt1/elasticsearch_backup/

OLD$ scp elasticsearch_backup.tgz user@NEW:/mnt1/dropbox (We are using
AmazonAWS server)

NEW$ tar -zxvf elasticsearch_backup.tgz

curl -XPUT 'http://localhost:9200/_snapshot/my_backup2' -d '{
"type": "fs",
"settings": {
"location": "/mnt1/dropbox/mnt1/my_backup/",
"compress": true
}
}'

^ (At that location, I see my indices directory that I've unzipped from tar
file that I've copied over from OLD server)

NEW$ curl -X POST
"localhost:9200/_snapshot/my_backup2/snapshot_1/_restore?wait_for_completion=true"

and the error message I get at this point is

{"error":"SnapshotMissingException[[my_backup2:snapshot_1] is missing];
nested:
FileNotFoundException[/mnt1/dropbox/mnt1/my_backup/snapshot-snapshot_1 (No
such file or directory)]; ","status":404}

It would be great if someone can point out what I can possibly do wrong here

Thanks.

Additionally, if I type

http://OLD:9200/_snapshot/my_backup/_all

I see

{

  • snapshots:
    [

    {
    - snapshot: "snapshot_1",
    - indices:
    [
    - "user_events_43294971_1401773217",
    - "user_events_111630610_1404969230",
    - "user_events_85270411_1398889365",
    - "user_events_86845965_1399787308"
    ],
    - state: "SUCCESS",
    - start_time: "2014-08-05T01:42:15.655Z",
    - start_time_in_millis: 1407202935655,
    - end_time: "2014-08-05T02:16:39.919Z",
    - end_time_in_millis: 1407204999919,
    - duration_in_millis: 2064264,
    - failures: [ ],
    - shards:
    {
    - total: 32,
    - failed: 0,
    - successful: 32
    }
    }
    ]

}

However, if I type

http://NEW:9200/_snapshot/my_backup2/_all

All I see is

{

  • snapshots: [ ]

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b1360def-15d8-4eab-8cbd-1257b56adca7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

1 Like
2

Hi,

Maybe it is permission issue on the new server directory?

Try to make chmod 777 on directory and then try to restore.

Thanks,

Dan.

3

Still the same issue here as well... Settings are the same on both servers,
but after copying (rsync) the files to the other server, the other server
fails to see any snapshots in the API call to list them. Same problem as
Donald: the respository has only the "indices" sub directory, and the PUT
command on the secondary server gives the same "(No such file or
directory)" error message.
Any help would be greatly appreciated!

-Bryan

On Tuesday, August 5, 2014 5:40:39 PM UTC-4, Donald Cha wrote:

I would like to use the Snapshot functionality to move indices from an OLD
server to a NEW server. My approach to do this task was to take a snapshot
from OLD server, copy the snapshot over to the NEW server (I zipped this
snapshot because it contained multiple indices), and try to restore that
snapshot from NEW server. However, I am not able to restore my snapshot on
NEW server.

Here are the commands that I've typed.

OLD$ curl -X PUT 'http://localhost:9200/_snapshot/my_backup' -d '{
"type" : "fs",
"settings" : {
"location" : "/mnt1/elasticsearch_backup/my_backup",
"compress" : true
}
}'

OLD$ curl -XPUT
"localhost:9200/_snapshot/my_backup/snapshot_1?wait_for_completion=true"
OLD$ tar cvfz elasticsearch_backup.tgz /mnt1/elasticsearch_backup/

OLD$ scp elasticsearch_backup.tgz user@NEW:/mnt1/dropbox (We are using
AmazonAWS server)

NEW$ tar -zxvf elasticsearch_backup.tgz

curl -XPUT 'http://localhost:9200/_snapshot/my_backup2' -d '{
"type": "fs",
"settings": {
"location": "/mnt1/dropbox/mnt1/my_backup/",
"compress": true
}
}'

^ (At that location, I see my indices directory that I've unzipped from
tar file that I've copied over from OLD server)

NEW$ curl -X POST
"localhost:9200/_snapshot/my_backup2/snapshot_1/_restore?wait_for_completion=true"

and the error message I get at this point is

{"error":"SnapshotMissingException[[my_backup2:snapshot_1] is missing];
nested:
FileNotFoundException[/mnt1/dropbox/mnt1/my_backup/snapshot-snapshot_1 (No
such file or directory)]; ","status":404}

It would be great if someone can point out what I can possibly do wrong
here

Thanks.

Additionally, if I type

http://OLD:9200/_snapshot/my_backup/_all

I see

{

  • snapshots:
    [

    {
    - snapshot: "snapshot_1",
    - indices:
    [
    - "user_events_43294971_1401773217",
    - "user_events_111630610_1404969230",
    - "user_events_85270411_1398889365",
    - "user_events_86845965_1399787308"
    ],
    - state: "SUCCESS",
    - start_time: "2014-08-05T01:42:15.655Z",
    - start_time_in_millis: 1407202935655,
    - end_time: "2014-08-05T02:16:39.919Z",
    - end_time_in_millis: 1407204999919,
    - duration_in_millis: 2064264,
    - failures: ,
    - shards:
    {
    - total: 32,
    - failed: 0,
    - successful: 32
    }
    }
    ]

}

However, if I type

http://NEW:9200/_snapshot/my_backup2/_all

All I see is

{

  • snapshots:

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4b7f8505-f4d2-466f-a0e0-aaac0821218c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

4

The permissions are wide open for the snapshot repository. I followed the
directions from the snapshot documentation; I wonder if I'm missing a
trailing slash or something, there are various files under different
indices underneath the sub directories beneath my
/snapshots/indices/{indexname}/ but there the "restore" command is looking
for them in the root of my "/snapshots" directory.

I will try with the index specific backups, but the round trip time to
determine whether or not a given PUT or POST work is about an half an hour
for each shot.. .So far, everything fails with the same 404 error, looking
for a file during the restore named /snapshots/snapshot-snapshot_1 that
really does not exist (in either the backup or the rsync destination).

Regards,
-Bryan

On Monday, August 11, 2014 4:49:57 PM UTC-4, Bryan Shannon wrote:

Still the same issue here as well... Settings are the same on both
servers, but after copying (rsync) the files to the other server, the other
server fails to see any snapshots in the API call to list them. Same
problem as Donald: the respository has only the "indices" sub directory,
and the PUT command on the secondary server gives the same "(No such file
or directory)" error message.
Any help would be greatly appreciated!

-Bryan

On Tuesday, August 5, 2014 5:40:39 PM UTC-4, Donald Cha wrote:

I would like to use the Snapshot functionality to move indices from an
OLD server to a NEW server. My approach to do this task was to take a
snapshot from OLD server, copy the snapshot over to the NEW server (I
zipped this snapshot because it contained multiple indices), and try to
restore that snapshot from NEW server. However, I am not able to restore my
snapshot on NEW server.

Here are the commands that I've typed.

OLD$ curl -X PUT 'http://localhost:9200/_snapshot/my_backup' -d '{
"type" : "fs",
"settings" : {
"location" : "/mnt1/elasticsearch_backup/my_backup",
"compress" : true
}
}'

OLD$ curl -XPUT
"localhost:9200/_snapshot/my_backup/snapshot_1?wait_for_completion=true"
OLD$ tar cvfz elasticsearch_backup.tgz /mnt1/elasticsearch_backup/

OLD$ scp elasticsearch_backup.tgz user@NEW:/mnt1/dropbox (We are using
AmazonAWS server)

NEW$ tar -zxvf elasticsearch_backup.tgz

curl -XPUT 'http://localhost:9200/_snapshot/my_backup2' -d '{
"type": "fs",
"settings": {
"location": "/mnt1/dropbox/mnt1/my_backup/",
"compress": true
}
}'

^ (At that location, I see my indices directory that I've unzipped from
tar file that I've copied over from OLD server)

NEW$ curl -X POST
"localhost:9200/_snapshot/my_backup2/snapshot_1/_restore?wait_for_completion=true"

and the error message I get at this point is

{"error":"SnapshotMissingException[[my_backup2:snapshot_1] is missing];
nested:
FileNotFoundException[/mnt1/dropbox/mnt1/my_backup/snapshot-snapshot_1 (No
such file or directory)]; ","status":404}

It would be great if someone can point out what I can possibly do wrong
here

Thanks.

Additionally, if I type

http://OLD:9200/_snapshot/my_backup/_all

I see

{

  • snapshots:
    [

    {
    - snapshot: "snapshot_1",
    - indices:
    [
    - "user_events_43294971_1401773217",
    - "user_events_111630610_1404969230",
    - "user_events_85270411_1398889365",
    - "user_events_86845965_1399787308"
    ],
    - state: "SUCCESS",
    - start_time: "2014-08-05T01:42:15.655Z",
    - start_time_in_millis: 1407202935655,
    - end_time: "2014-08-05T02:16:39.919Z",
    - end_time_in_millis: 1407204999919,
    - duration_in_millis: 2064264,
    - failures: ,
    - shards:
    {
    - total: 32,
    - failed: 0,
    - successful: 32
    }
    }
    ]

}

However, if I type

http://NEW:9200/_snapshot/my_backup2/_all

All I see is

{

  • snapshots:

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e0de73d5-8e3d-4ab7-ab6f-266c72fa437a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

5

If you created snapshot on filesystem, you need to copy backup from the
master node VM of your cluster. This is where entire snapshot is located.

On Wednesday, 6 August 2014 03:10:39 UTC+5:30, Donald Cha wrote:

I would like to use the Snapshot functionality to move indices from an OLD
server to a NEW server. My approach to do this task was to take a snapshot
from OLD server, copy the snapshot over to the NEW server (I zipped this
snapshot because it contained multiple indices), and try to restore that
snapshot from NEW server. However, I am not able to restore my snapshot on
NEW server.

Here are the commands that I've typed.

OLD$ curl -X PUT 'http://localhost:9200/_snapshot/my_backup' -d '{
"type" : "fs",
"settings" : {
"location" : "/mnt1/elasticsearch_backup/my_backup",
"compress" : true
}
}'

OLD$ curl -XPUT
"localhost:9200/_snapshot/my_backup/snapshot_1?wait_for_completion=true"
OLD$ tar cvfz elasticsearch_backup.tgz /mnt1/elasticsearch_backup/

OLD$ scp elasticsearch_backup.tgz user@NEW:/mnt1/dropbox (We are using
AmazonAWS server)

NEW$ tar -zxvf elasticsearch_backup.tgz

curl -XPUT 'http://localhost:9200/_snapshot/my_backup2' -d '{
"type": "fs",
"settings": {
"location": "/mnt1/dropbox/mnt1/my_backup/",
"compress": true
}
}'

^ (At that location, I see my indices directory that I've unzipped from
tar file that I've copied over from OLD server)

NEW$ curl -X POST
"localhost:9200/_snapshot/my_backup2/snapshot_1/_restore?wait_for_completion=true"

and the error message I get at this point is

{"error":"SnapshotMissingException[[my_backup2:snapshot_1] is missing];
nested:
FileNotFoundException[/mnt1/dropbox/mnt1/my_backup/snapshot-snapshot_1 (No
such file or directory)]; ","status":404}

It would be great if someone can point out what I can possibly do wrong
here

Thanks.

Additionally, if I type

http://OLD:9200/_snapshot/my_backup/_all

I see

{

  • snapshots:
    [

    {
    - snapshot: "snapshot_1",
    - indices:
    [
    - "user_events_43294971_1401773217",
    - "user_events_111630610_1404969230",
    - "user_events_85270411_1398889365",
    - "user_events_86845965_1399787308"
    ],
    - state: "SUCCESS",
    - start_time: "2014-08-05T01:42:15.655Z",
    - start_time_in_millis: 1407202935655,
    - end_time: "2014-08-05T02:16:39.919Z",
    - end_time_in_millis: 1407204999919,
    - duration_in_millis: 2064264,
    - failures: ,
    - shards:
    {
    - total: 32,
    - failed: 0,
    - successful: 32
    }
    }
    ]

}

However, if I type

http://NEW:9200/_snapshot/my_backup2/_all

All I see is

{

  • snapshots:

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/cb11f337-b85e-4dbe-8777-94283b409c48%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.