How to move a 4 node managed cluster to a new datacenter (basic license)

Elastic Stack Elasticsearch
1

I was hoping someone could advise me with the following problem.

I have a four node Linux cluster with a basic licence in a managed datacenter. All nodes have the same hardware resources 1TB SSD. At this point in time the master node has reached 95% usage of the storage space. Seeing its not possible to increase the storage on the servers, I would like to move the cluster to my servers in a different datacenter. However the basic license as I understand does not support Cross Cluster Replication only snapshots. Snapshots can only do remote to S3, Azure etc. Doing 4TB does not seem like an option. Any suggestions?

2

Snapshots also support a shared file system.
You have to setup a NFS / SMB drive shared across all the nodes and setup a repository.
The documentation is here.

Once the snapshot is done, you can move the whole content of the shared drive to the new datacenter, setup a read only repository on the new cluster and restore from there.

3

Thanks for the quick answer.

Would this method work despite the 4 nodes having low disk space. E.g. not that the snapshot itself requires diskspace on the host when writing to the NFS/SMB drive?

And also would the NFS/SMB method work over the WAN seeing storage would be outside of the datacenter / in the new datacenter?

4

It should work as it will just checkpoint the files in your path.data and copy them over network.

The first objective is to snapshot your data in the current network to the NFS/SMB.

Once you have the snapshot, you have 2 options:

  1. Mount the same NFS/SMB on the destination cluster as a repository (if the NFS/SMB is mountable by both data centers)
  2. Tar.gz the whole NFS/SMB drive, move it to the other data center, then:
    1. You can untar the files on a HTTP server (serving the files) and setup a read-only URL repository (doc) and restore from it
    2. You can untar the files on a NFS/SMB drive on the new data center, setup a fs repository on the new cluster and restore
5

Thanks for the answer, is there anything that can be said about the size of the snapshot. In this case the 4 nodes only have the ubuntu os and elasticsearch installed each node has 1TB of storage and are at 95%.

6

The snapshot will take more or less the size of the sum of the primary shards.

7

Thanks for the good advice. Added a samba box, cluster is now snapshotting. Used this article for extra help. With my basic knowledge of Ubuntu it's doing what it should.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.