Cluster replication

(Rabeet waqar) #1

Is it possible to create the replica of whole cluster?

Mysql vs Elasticsearch data replication?
(Joshua Rich) #2

Hi @rabeetwaqar,

Can you describe what you are trying to solve by replicating your entire cluster? DR? Backups?

(Rabeet waqar) #3

Yes backups. For any reason if my hardware fails i should have a backup of my whole cluster. Is it possible to create?

(Magnus Bäck) #4

What kind of hardware failure do you want to protect against? With a multi-node cluster with shard replicas you're pretty resilient to begin with.

Also, see this recent topic:

(Rabeet waqar) #5

Thanks magnus for quick response
Sure i will look into it.

(Rabeet waqar) #6

Hello @magnusbaeck ,

I have created a repository and successfully managed to create a snapshot of my index but if we run a Cron job lets assume :

  • Server A(SA)
  • Server B(SB)

My cluster is created on SA and i replicate/backup it using snapshot to SB.

My question is if my cluster has more than 50,0000 data and i replicate it after every 2 minutes , the SB will not be able to manage it or bandwidth will be consumed or it will eventually get slow.
What if my SA gets down , then the domain will be automatically changed but the data with in SB cluster must be the same as SA cluster was containing before crash.

1- Can we replicate SA cluster difference to SB. Note : Only the differences i.e indexes and shards maybe.
2- And if yes what would be the exact procedure to do this?

Thanks in advance

(Magnus Bäck) #7

The snapshot/restore mechanism won't work well for real-time failover situations. But why not just have your two hosts join the same cluster? With one replica of each index it's okay if one node drops out.

(Rabeet waqar) #8

What if the server gets down which is containing the cluster? Then both dependencies will get down.

(Magnus Bäck) #9

What do you mean? The whole point of an ES cluster is that no single node contains all state and that (given sufficient index replicas) any machine can drop out of the cluster without affecting its operations.

(Rabeet waqar) #10


How can i join two elasticsearch instances running on different servers to a single host?
Should i update my .yml files?

(Magnus Bäck) #11

Do you have a two-node cluster that you want to turn into a one-node cluster? Or two separate clusters where you want to migrate all data from one to the other?

(Rabeet waqar) #12

Thanks for prompt reply,

Yes the second case i'm looking for. I want to replicate cluster data running on different servers.

What steps should i follow?

(Magnus Bäck) #13

If it's a one time thing look into the snapshot/restore mechanism that you've already explored. If you're talking about continuous replication then we're discussing in circles and I'll refer you to what I've said previously in this thread.

(Rabeet waqar) #14

Yes i'm looking for continuous replication.
Okay thanks.

(system) #15