# CCR - Remove follow\_stats and License fetching problem
**URL:** https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389
**Category:** Elasticsearch
**Tags:** ccr-cross-cluster-replication
**Created:** [April 4, 2019, 10:40am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389 "2019-04-04T10:40:00Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@kley](https://discuss.elastic.co/u/kley)
#### Post date: [April 4, 2019, 10:40am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/1 "2019-04-04T10:40:01Z")
Hi!
I'm playing around with the new cross cluster replication feature and ran into some issues. Firstly, it worked like a charm, data got replicated and everything was good. After that, I decided to put load on the leader index to see if the follower can catch up - spoiler, with default settings it couldn't.
Anyway, now I got the issue, that the follower index doesn't replicate any data anymore.
When i call \_ccr/stats i get this message:
```auto
{
"leader_index": "cluster2-follows-cluster1",
"timestamp": 1554360455200,
"auto_follow_exception": {
"type": "exception",
"reason": "ElasticsearchStatusException[can not fetch remote cluster state as the license state of the remote cluster [elasticsearch] could not be determined]; nested: ResourceNotFoundException[license info is missing for cluster [elasticsearch]];",
"caused_by": {
"type": "status_exception",
"reason": "can not fetch remote cluster state as the license state of the remote cluster [elasticsearch] could not be determined",
"caused_by": {
"type": "resource_not_found_exception",
"reason": "license info is missing for cluster [elasticsearch]"
}
}
}
}
]
```
can this be the root cause why my data doesn't get replicated anymore?
Furthermore, when I call \_ccr/stats i get the info about an Index I used to follow (visit). This index doesn't exist anymore but i the follower still wants to fetch data from it - resulting in a index not found exception. This was achieved with the auto-follow functionality. Is there a way to remove the "knowledge" of that index? It will never come back and I want to get rid of those exceptions.
The response looks something like this:
```auto
"follow_stats": {
"indices": [
{
"index": "ccr-visit",
"shards": [
{
"remote_cluster": "elasticsearch",
"leader_index": "visit",
"follower_index": "ccr-visit",
"shard_id": 1,
"leader_global_checkpoint": -1,
...
}
}
}
```
Thanks in advance!
And sorry, for the formatting,... I don't find formatting options anywhere ☹
---
### Author:  [@ywelsch](https://discuss.elastic.co/u/ywelsch)
#### Post date: [April 5, 2019, 7:51am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/2 "2019-04-05T07:51:45Z")
> can not fetch remote cluster state as the license state of the remote cluster [elasticsearch] could not be determined
If you're using a trial license, it may be that the trial license on the leader cluster expired.
> Is there a way to remove the "knowledge" of that index?
yes, you can turn a follower index into a regular index (see [Unfollow API | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/ccr-post-unfollow.html) ). From that point on, you can also locally write into that index just as any other index.
---
### Author:  [@kley](https://discuss.elastic.co/u/kley)
#### Post date: [April 5, 2019, 8:05am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/3 "2019-04-05T08:05:52Z")
Hi again 🙂
I can tell for sure, that the trial is not expired - i started the trial on both clusters 3 days ago at the same time.
Regarding converting a follower index into a regular one. Both indices don't exist anymore and will never come back - was just for testing. Fun fact, I still get this exception when I create a leader index starting with visit-\*. But, the leader index has a totally different name then the one mentioned in the exception.
```auto
[2019-04-05T09:55:12,683][WARN][o.e.x.c.a.ShardFollowTasksExecutor] [node-2] [ccr-visit-...][2] background management of retention lease [elasticsearch/ccr-visit-.../H9bsXxlRTcSQzmqtVAUQYg-following-elasticsearch/visit-.../bBwfwk3JTIyPx0XwMvgkzA] failed while following
org.elasticsearch.index.IndexNotFoundException: no such index
```
On the setup - to the left, the leader index - to the right, the follower
As you can see, the follower did not replicate any data, only created the index. The above warning occurs in the logs and i get the failed licence fetch message.
I delted the auto follow pattern and readded it, I restarted elasticsearch (yea i know) and it didn't help as well.
If you need any other logs or settings i used for this setup let me know.
---
### Author:  [@ywelsch](https://discuss.elastic.co/u/ywelsch)
#### Post date: [April 5, 2019, 10:10am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/4 "2019-04-05T10:10:08Z")
Can you privately share the output of `GET /_xpack/license` of both clusters?
The `license info is missing for cluster [elasticsearch]` message is very surprising.
---
### Author:  [@kley](https://discuss.elastic.co/u/kley)
#### Post date: [April 8, 2019, 6:03am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/5 "2019-04-08T06:03:59Z")
Hmm, ok, I called the license API and that's the response of cluster 1 ( node-1 = leader node):
```auto
{
"license": {
"status": "active",
"uid": "e6ce6927-1247-4f31-a98e-...",
"type": "trial",
"issue_date": "2019-04-02T08:21:22.878Z",
"issue_date_in_millis": 1554193282878,
"expiry_date": "2019-05-02T08:21:22.878Z",
"expiry_date_in_millis": 1556785282878,
"max_nodes": 1000,
"issued_to": "elasticsearch",
"issuer": "elasticsearch",
"start_date_in_millis": -1
}
}
```
which looks good, but cluster 2 (node-2 = follower node) looks not so healthy:
```auto
{
"error": {
"root_cause": [
{
"type": "parse_exception",
"reason": "failed to parse licenses expected field"
}
],
"type": "parse_exception",
"reason": "failed to parse licenses expected field"
},
"status": 400
}
```
---
### Author:  [@ywelsch](https://discuss.elastic.co/u/ywelsch)
#### Post date: [April 9, 2019, 2:49pm UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/6 "2019-04-09T14:49:26Z")
If this is a cluster for experimentation purposes only, perhaps remove / delete the data folder of node-2 and start afresh?
---
### Author:  [@kley](https://discuss.elastic.co/u/kley)
#### Post date: [April 10, 2019, 11:33am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/7 "2019-04-10T11:33:46Z")
Yannick, we did it!
So, I deleted the (follower)node folder, which is ok because it's a test environment, had to re-activate the trial and it started working again.
Nevertheless, this will be a big nono in production. Is there maybe anything more I can share for you to maybe identify the root cause of this?
Thanks so far!
---
### Author:  [@ywelsch](https://discuss.elastic.co/u/ywelsch)
#### Post date: [April 11, 2019, 9:46am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/8 "2019-04-11T09:46:54Z")
The symptoms and exception message suggest an expired trial license. If you still have the original data folder, you can privately share the `data/nodes/0/_state/global-*.st` file. Otherwise, if this occurs again, please run the start trial command to see if there is already a trial license in place, see [https://www.elastic.co/guide/en/elasticsearch/reference/current/start-trial.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/start-trial.html)
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 9, 2019, 9:46am UTC](https://discuss.elastic.co/t/ccr-remove-follow-stats-and-license-fetching-problem/175389/9 "2019-05-09T09:46:55Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.