Elasticsearch cross cluster data migration approach

Elastic Stack Elasticsearch
1

Hi,

We have an existing cluster running ES 5.6.7 version. We want to create a new cluster running ES 7.17.5 version and migrate all data to this cluster.

We have close to 300 Million documents present on the existing cluster.

We are confused between two approaches :

  • Reindex From Remote

  • Snapshot and Restore

What do you guys think is a better approach considering below factors :

  • We want zero downtime. Will Reindex from Remote affect performance of existing cluster ?

  • Which is faster and if anyone can give me a rough estimate on how much time each process will take ?

  • For snapshot and restore, we need to create a temporary 6.x cluster, reindex and then again snapshot and restore to ES 7 cluster.

Please suggest if there is any other alternative approach apart from these two.

2

Given it's a query to the original cluster it'll have some impact, yes.

I would use reindex from remote due to the changes in mappings between 5.X and later versions.

I would be looking to go to 8.X to be honest. Otherwise you're looking at another migration down the path.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.