How to sync elasticsearch data between DC, DR

Elastic Stack Elasticsearch
1

Hi Team,

We have elasticsearch running on our DC side so standalone as of now.

We would like to know how can we sync elasticsearch data to DR side so that if DR happens, we can still see data in kibana.

It is installed on on-premise environment on virtual machines.

Thanks,

2

Welcome to our community! :smiley:

CCR and CCS are designed for these, check out Set up a cluster for high availability | Elasticsearch Guide [7.16] | Elastic

3

Hi @Warkolm,

CCR comes in paid license. CCS although seems to be in free license but it looks more for searching on multiple clusters rather than doing replication.

So removing CCS from the discussion for now.

Can you or anyone suggest replication for open source solution.

Thanks,

4

Potentially your could use logstash to pull from one Elasticsearch and push to another. I don't know how efficient it would be but that's my initial thought.

5

Hi @legoguy1000, Thanks. Do you have reference document for this?

6

Elasticsearch input plugin | Logstash Reference [7.16] | Elastic to query and then an Elasticsearch output to the other cluster. What else u need, idk but that would be the start.

7

I do not think I have ever seen this used successfully. The standard method before CCR was available was to feed both clusters in parallel based on a message queue, e.g. Kafka. Eavh cluster would be indexed into using a dedicated Logstash instance. That comes with its own set if challenges, but I have seen that used at scale in the past.

It is a difficult problem to solve properly, which is why I suspect it is an Enterprise feature with a commercial licence.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.