Elasticsearch data migration

Elastic Stack Elasticsearch
1

Hi

I need to migrate data from Elasticsearch on premises to Elasticsearch Cloud
Is there any documentation I can refer??

2

I know that we are thinking of providing such a guide. I don't think it's available yet though.

I can think of multiple solutions:

  • Reindex from the source the same way you indexed in your local cluster
  • Create a job which use the Scroll API against the local cluster, get the _source and send it as part of a bulk request to the cloud cluster.
  • Use Snapshot feature to store the data within AWS, Azure or GCP. And then use Restore to restore from the cloud cluster. See https://www.elastic.co/guide/en/cloud/current/ec-custom-repository.html

I hope this can help.

3

Maybe also some logstash with elasticsearch input and elasticseach output that bind your 2 clusters ?
We do that sometime for testing purpose.

1 Like
4

Yeah! Very good idea indeed. It's better than my solution 2 as you won't have to write the code yourself :slight_smile:

5

Yes, but maybe compare size & doc number of both indices at the end, so you'll be sure that there's no data loss

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.