# CCR - auto-follow problem on data streams

**URL:** https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789
**Category:** Elasticsearch
**Tags:** ccr-cross-cluster-replication
**Created:** [December 7, 2023, 8:26am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789 "2023-12-07T08:26:55Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Wojciech\_Kwiecien](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wojciech_kwiecien/32/109829_2.png) [@Wojciech\_Kwiecien](https://discuss.elastic.co/u/Wojciech_Kwiecien)
#### Post date: [December 7, 2023, 8:26am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789/1 "2023-12-07T08:26:55Z")

</div>

I found this topic [Elastic Cross Cluster Replication of Data Stream](https://discuss.elastic.co/t/elastic-cross-cluster-replication-of-data-stream/346178)

But I do not find a solution for me.  
I was able to create CCR auto-follow pattern and nothing happened when I ran GET /\_ccr/stats  
After I follow instructions from [Elastic Cross Cluster Replication of Data Stream](https://discuss.elastic.co/t/elastic-cross-cluster-replication-of-data-stream/346178)  
and after rollover on source POST traces-apm-default/\_rollover something happened.  
I received few errors :

```auto
{ "auto_follow_stats" : { "number_of_failed_follow_indices" : 134, "number_of_failed_remote_cluster_state_requests" : 0, "number_of_successful_follow_indices" : 0, "recent_auto_follow_errors" : [{ "leader_index" : "traces:.ds-traces-apm-default-2023.12.06-000092", "timestamp" : 1701868421437, "auto_follow_exception" : { "type" : "exception", "reason" : "java.lang.IllegalArgumentException: cannot follow backing index [.ds-traces-apm-default-2023.12.06-000092], because local data stream [traces-apm-default] is no longer marked as replicated", "caused_by" : { "type" : "illegal_argument_exception", "reason" : "cannot follow backing index [.ds-traces-apm-default-2023.12.06-000092], because local data stream [traces-apm-default] is no longer marked as replicated" } } } ]

```

In another case what I tried to do was to add suffixes but after that I received

```auto
"recent_auto_follow_errors" : [
{
"leader_index" : "follower:.ds-traces-apm-default-2023.12.06-000094",
"timestamp" : 1701874886251,
"auto_follow_exception" : {
"type" : "exception",
"reason" : "java.lang.IllegalArgumentException: a backing index name in the local and remote cluster must remain the same",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "a backing index name in the local and remote cluster must remain the same"
}
}
}
],

If someone can take a look and help I will be very grateful.

```

---

<div class="post-metadata">

### Author: ![Wojciech\_Kwiecien](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wojciech_kwiecien/32/109829_2.png) [@Wojciech\_Kwiecien](https://discuss.elastic.co/u/Wojciech_Kwiecien)
#### Post date: [December 7, 2023, 8:36am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789/2 "2023-12-07T08:36:17Z")

</div>

@adsandie Can you take a look and help 🙂

---

<div class="post-metadata">

### Author: ![yago82](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yago82/32/97755_2.png) [@yago82](https://discuss.elastic.co/u/yago82)
#### Post date: [December 7, 2023, 8:41am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789/3 "2023-12-07T08:41:50Z")

</div>

Hi,

The first error says that the local data stream 'traces-apm-default' isn't flagged as replicated anymore. That could happen if the data stream got deleted or changed in a way that messes up the replication process. Check up on its status and make sure it's set up properly for replication.

The second error points out that the names of the indexes in both the local and remote clusters need to match. So, when you're setting up Cross-Cluster Replication (CCR), it's super important that the index names in the main and follower clusters are exactly the same. If you've added a tag or something to the index name in the follower cluster, that might be causing the issue. Just double-check to ensure both clusters have the same index names when you're setting up CCR.

Regards

---

<div class="post-metadata">

### Author: ![Wojciech\_Kwiecien](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wojciech_kwiecien/32/109829_2.png) [@Wojciech\_Kwiecien](https://discuss.elastic.co/u/Wojciech_Kwiecien)
#### Post date: [December 7, 2023, 9:18am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789/4 "2023-12-07T09:18:43Z")

</div>

![image](https://us1.discourse-cdn.com/elastic/original/3X/6/f/6f4d45babe1f113527cf6d5ee209cdd4c25e48de.png)  
When I set it up this way and ran GET /traces-apm-default/\_ccr/info, nothing was visible in the logs. When I clicked [View Follower Indexes in Index Management], no indexes were created on the target cluster.

I thought that after setting it up, it would take a while to create indexes and start replicating data, but I waited a few hours and nothing happened.

The only action in the visible in logs was when index was rollover on the source cluster.

Is there any option to check the progress of this process?

Do you know how long this process might take? There are 3 indexes in the source cluster, each 50 GB in size

---

<div class="post-metadata">

### Author: ![Wojciech\_Kwiecien](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wojciech_kwiecien/32/109829_2.png) [@Wojciech\_Kwiecien](https://discuss.elastic.co/u/Wojciech_Kwiecien)
#### Post date: [December 7, 2023, 12:38pm UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789/5 "2023-12-07T12:38:56Z")

</div>

Our version is 7.17.12 for both.  
Both cluster have the same settings in traces-apm@custom in component templates

```auto
{
  "index": {
    "refresh_interval": "30s",
    "number_of_shards": "1",
    "auto_expand_replicas": "0-0",
    "number_of_replicas": "0"
  }
}

```

---

<div class="post-metadata">

### Author: ![adsandie](https://avatars.discourse-cdn.com/v4/letter/a/8c91f0/32.png) [@adsandie](https://discuss.elastic.co/u/adsandie)
#### Post date: [December 11, 2023, 1:13am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789/6 "2023-12-11T01:13:10Z")

</div>

There should be no data stream on the destination cluster. If there is, you must delete it.

If you run GET \_data\_stream/traces-apm-default, you will see "replicated": false. For data stream ccr, you cannot do a snapshot/restore a portion of the data stream and do a ccr.

Another option for you if you do not want to delete the destination data stream is to append a suffix in your auto-follow pattern like "-replicated".

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 8, 2024, 1:13am UTC](https://discuss.elastic.co/t/ccr-auto-follow-problem-on-data-streams/348789/7 "2024-01-08T01:13:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
