# Cross cluster replication

**URL:** <https://discuss.elastic.co/t/cross-cluster-replication/308735>\
**Category:** Elasticsearch\
**Tags:** ccr-cross-cluster-replication\
**Created:** [July 3, 2022, 1:55am UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735 "2022-07-03T01:55:52Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nicole-Ann\_Menezes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicole-ann_menezes/32/107855_2.png) [@Nicole-Ann\_Menezes](https://discuss.elastic.co/u/Nicole-Ann_Menezes)\
**Post date:** [July 3, 2022, 1:55am UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/1 "2022-07-03T01:55:52Z")

</div>

Exploring CCR.

Flow - fluentd----\>Load balancer--'--\>elasticsearch ccr\<---kibana.

Cluster A and Cluster B are set up with CCR. As per the documentation-An index has many shards, the follower replicas are in another cluster than the leader.

My question is if Cluster A has a primary index and Cluster A is down can fluentd still write to a new shard for the same index on Cluster B ?

Aa each Index can have 5 primary shards per index so i am assuming that Elasticsearch can automatically create a new primary shard and then fluentd can continue to write to elasticsearch?

Or is it that all primary index only exists in Cluster A? So if cluster A is down fluentd cannot write to that index as you cannot write to a follower shard?however a user using kibana can still search that index?

If we cannot write than how can this be considered HA?

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 3, 2022, 6:05am UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/2 "2022-07-03T06:05:44Z")

</div>

> [@Nicole-Ann\_Menezes](#):
>
> Cluster A and Cluster B are set up with CCR. As per the documentation-An index has many shards, the follower replicas are in another cluster than the leader.

The leader index in cluster A has both primaries and replicas and can be written to. If you use CCR to replicate this to cluster B the follower index in cluster B will also have primary and replica shards but be read-only.

> [@Nicole-Ann\_Menezes](#):
>
> My question is if Cluster A has a primary index and Cluster A is down can fluentd still write to a new shard for the same index on Cluster B ?

No.

> [@Nicole-Ann\_Menezes](#):
>
> Aa each Index can have 5 primary shards per index so i am assuming that Elasticsearch can automatically create a new primary shard and then fluentd can continue to write to elasticsearch?

No, that is not the case.

> [@Nicole-Ann\_Menezes](#):
>
> If we cannot write than how can this be considered HA?

For immutable data CCR is often set up [as bi-directional](https://www.elastic.co/guide/en/elasticsearch/reference/7.17/xpack-ccr.html#ccr-bi-directional-replication) using two separate indices and you write to a cluster specific index. If you want to write to a single index name I wonder if you might be able to use a ingest pipeline per index that change the index name to the appropriate cluster specific name.

---

<div class="post-metadata">

**Author:** ![Nicole-Ann\_Menezes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicole-ann_menezes/32/107855_2.png) [@Nicole-Ann\_Menezes](https://discuss.elastic.co/u/Nicole-Ann_Menezes)\
**Post date:** [July 3, 2022, 11:54am UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/3 "2022-07-03T11:54:03Z")

</div>

Thanks for your reply.

Regarding your response " For immutable data CCR is often using two separate indices and you write to a cluster specific index"

I think that could be a possible solution...

From the start fluentd/ elasticsearch can be configured to write to either cluster specific index. Assuming round robin.  
For example cluster A indices is Log03072022.1and then on Cluster B Log03072022.2. Both are part of an index pattern log\*. If cluster A is down then fluentd continues to write to a Load balancer who is now only pointing to cluster B. Hence the HA is achieved??

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 3, 2022, 12:21pm UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/4 "2022-07-03T12:21:54Z")

</div>

I was thinking the following. Will use sample names, so please change this for som ething better if you try it out. Set up fluentd to index into the `logdata` alias/index. This can round robin across both clusters or have one as preference and only failover if required.

In cluster A set up a `logdata_A` data stream that you replicate to cluster B using CCR. In cluster B set up a `logdata_B` data stream that you replicate to cluster A using CCR.  
As you want to write to the same index name on both cluster you will need a way to redirect writes. I have not tried the following suggestions so am not sure they will work or not.

- Create a local alias `logdata` in each cluster and pouint this to the local data stream.
- Create a local dummy index named logdata in each cluster and associate this with a default ingest pipeline that changes the index name of all documents written to it to the local data stream.

You can then create an index pattern `logdata_*` to access all these indices in one or the cluster.  
I suspect you should be able to now create an index named

---

<div class="post-metadata">

**Author:** ![Nicole-Ann\_Menezes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicole-ann_menezes/32/107855_2.png) [@Nicole-Ann\_Menezes](https://discuss.elastic.co/u/Nicole-Ann_Menezes)\
**Post date:** [July 3, 2022, 12:56pm UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/5 "2022-07-03T12:56:43Z")

</div>

Thanks again for your reply. Yes i think the alias is a better option will have to now test the theory:) merci

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [July 3, 2022, 3:19pm UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/6 "2022-07-03T15:19:23Z")

</div>

@Nicole-Ann_Menezes Here are some pictures of what @Christian_Dahlqvist was describing...in short in each cluster you write to the Leader and read from both the Leader and Follower... These are just high level obviously there are many details but hope this helps a bit.

Here are a couple architecture diagrams to think about... The first is more what you two are describing...

 ![Screen Shot 2022-07-03 at 8.13.18 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/7/17dcc86bbc7a937d57665c9ac518119c5e9feded.png)

And a Failed Cluster Scenario

 ![Screen Shot 2022-07-03 at 8.15.39 AM](https://us1.discourse-cdn.com/elastic/original/3X/5/4/543585caa74f343752e9cc6c50db73b5bbe1ee16.png)

And a different more local architecture as well...

 ![Screen Shot 2022-07-03 at 8.02.43 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/1/115705e4e466cea42932934513f1d5faf6475555.png)

Then from there you can work on your failure scenarios ... like the one you mentioned...

 ![Screen Shot 2022-07-03 at 8.03.12 AM](https://us1.discourse-cdn.com/elastic/original/3X/e/4/e4200203df360e044c1dcf6f7ff64e4d51de8386.png)

---

<div class="post-metadata">

**Author:** ![Nicole-Ann\_Menezes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nicole-ann_menezes/32/107855_2.png) [@Nicole-Ann\_Menezes](https://discuss.elastic.co/u/Nicole-Ann_Menezes)\
**Post date:** [July 3, 2022, 4:23pm UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/7 "2022-07-03T16:23:36Z")

</div>

@stephenb thanks this really will help. I have a meeting with an elasticsearch rep on the 5th july to discuss licensing options as we want to run these on K8S. Hoping all lines up. 🙂

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 31, 2022, 4:24pm UTC](https://discuss.elastic.co/t/cross-cluster-replication/308735/8 "2022-07-31T16:24:10Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.