How does Cross Cluster Replication work during a DR failover?

Elastic Stack Elasticsearch
1

Happy Scenario
We have setup ES on two data-centers - A and B.
Data is coming to A from Kafka/Spark combination hosted in A too.
ES in B has a follower index.
ES in A has a leader index.
Both A and B elastic clusters get read-traffic from clients local to their data-centers.

DR setup
For DR-failover, Kafka/Spark are present in B as well and they get written to only if A becomes un-available.

Now during failover, B would begin to get writes from their local Kafka/Spark and not from A.

After DR
Once the DR is fixed, it is expected that things would get restored to the happy-scenario.
But how do we replicate the changes from B to A that occurred during the DR failover?
We have inserts as well as updates.

Does ES support bi-directional CCR or anything else that can help to make the DR switching operations automatic?

2

A bit more information about your use case would be useful. Are you using time based indices? Do you update or delete data or just insert new documents?

3

are you using two different clusters?
if so, design a solution to capture all transactions during DR window and process it once the DR window is over,

4

We have both kinds of clusters actually and will appreciate appropriate information for both (time-based indices and non-time-based indices)

5

@Christian_Dahlqvist , can you help here please?

6

True bi-directional replication for a single index is as far as I know not supported, so I suspect indices that are updated will need to be replicated one way and require manual intervention to reverse if the initial cluster fails. For data that is just indexed you can set up two set of indices, one in each site, and replicate to the other site. This is referred to as bi-directional replication in this blog postYou then write to the appropriate index in each cluster.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.