Backup from ES data to a remote machine

Elastic Stack Elasticsearch
1

hi all...
i am working on a project that puts its results to elastic search during running on a server... my data is growing and i want to save my elastic search data in a new remote server in order to move whole data from entire server to new remote server and continue there.
As i read, i should create repository and take snapshot...i did it locally on my entire server by creating repository, setting path.repo in .yml and taking snapshot!
The problem is that, i want to backup my data on a remote server (not locally) in order to continue on it...
i should say that, i am very new at elastic search world, so please guide me in details...
i need your fast replies, thanks a lot...
Best regards...

2

I have been succesfully using elasticdump to move indices from cluster to cluster or to create daily backups of my elasticsearch data

3

Reindex API can read old cluster and index in the new cluster

4

@dadoonet thanks for your reply... i used Reindex API... it seems that there is a problem about some data types, because i faced with this error and reindex process is not completed:

"error" : {
"root_cause" : [
{
"type" : "status_exception",
"reason" : "body={"error":{"root_cause":[{"type":"search_context_missing_exception","reason":"No search context found for id [41875]"},{"type":"search_context_missing_exception","reason":"No search context found for id [41874]"},{"type":"search_context_missing_exception","reason":"No search context found for id [41870]\ ...........
}
there is different types of data in my results such as string, date, id , .... does it relate to this error?
i have another question; my data is very large, more than 50 G... is there any limit about size in remote Reindex method?

5

Most likely you got a timeout or something while running the scroll.

my data is very large, more than 50 G.

That might explain.

It depends on the size of your docs. As per documentation, I can read:

Reindexing from a remote server uses an on-heap buffer that defaults to a maximum size of 100mb. If the remote index includes very large documents you’ll need to use a smaller batch size. The example below sets the batch size 10 which is very, very small.

May be change the size. Or look at slicing options???

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.