# ES Cross Cluster Replication Issue

**URL:** <https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950>\
**Category:** Elasticsearch\
**Tags:** ccr-cross-cluster-replication\
**Created:** [November 7, 2019, 10:55am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950 "2019-11-07T10:55:59Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sidsniper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidsniper/32/57320_2.png) [@sidsniper](https://discuss.elastic.co/u/sidsniper)\
**Post date:** [November 7, 2019, 10:55am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950/1 "2019-11-07T10:55:59Z")

</div>

Hi,

I am getting below error while trying to setup ccr:

"fatal\_exception" : {  
"type" : "es\_rejected\_execution\_exception",  
"reason" : "rejected execution of org.elasticsearch.common.util.concurrent.TimedRunnable@1e0fb72 on QueueResizingEsThreadPoolExecutor[name = localhost/search, queue capacity = 1000, min queue capacity = 1000, max queue capacity = 1000, frame size = 2000, targeted response rate = 1s, task execution EWMA = 54.1ms, adjustment amount = 50, org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor@4164846[Running, pool size = 7, active threads = 7, queued tasks = 1090, completed tasks = 1734719]]"

Can someone please help me since I am new with this.

ES Version: 6.8

Thanks,  
Sid

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [November 16, 2019, 12:45am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950/2 "2019-11-16T00:45:40Z")

</div>

This means that the search queue on your remote (where the follower shard is pulling from) is at capacity, and rejected the search requests the follower runs on the remote cluster to find the changes that it needs to replicate. This means that your remote cluster is overloaded.

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [November 16, 2019, 1:06am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950/3 "2019-11-16T01:06:23Z")

</div>

I opened [https://github.com/elastic/elasticsearch/pull/49213](https://github.com/elastic/elasticsearch/pull/49213) so that this is no longer considered fatal.

---

<div class="post-metadata">

**Author:** ![sidsniper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidsniper/32/57320_2.png) [@sidsniper](https://discuss.elastic.co/u/sidsniper)\
**Post date:** [November 19, 2019, 5:26am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950/4 "2019-11-19T05:26:40Z")

</div>

Thanks @jasontedor. I wanted to understand one more thing. Currently we have an ELB in front of Elastic Data nodes (ELB:9200). For replication, we need to apply cluster settings something like below:  
curl -X PUT "10.167.65.123:9200/\_cluster/settings?pretty" -H 'Content-Type: application/json' -d'  
{  
"persistent" : {  
"cluster" : {  
"remote" : {  
"us\_east\_cluster" : {  
"seeds" : [  
" **10.24.21.168:9300**"  
]  
}  
}  
}  
}  
}  
'

So can we use ELB:9300 for seeds instead of node IP's since they are not reliable? Currently I am not sure, if the replication will break if the node IP's get refreshed within the cluster.

Thanks,  
Sid

---

<div class="post-metadata">

**Author:** ![sidsniper](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sidsniper/32/57320_2.png) [@sidsniper](https://discuss.elastic.co/u/sidsniper)\
**Post date:** [December 4, 2019, 4:38am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950/5 "2019-12-04T04:38:07Z")

</div>

@jasontedor can you please help here with my above query?

---

<div class="post-metadata">

**Author:** ![jasontedor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jasontedor/32/66992_2.png) [@jasontedor](https://discuss.elastic.co/u/jasontedor)\
**Post date:** [December 10, 2019, 2:17am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950/6 "2019-12-10T02:17:00Z")

</div>

Why are your node IP's changing? Can you refer to them consistently by name?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 7, 2020, 2:21am UTC](https://discuss.elastic.co/t/es-cross-cluster-replication-issue/206950/7 "2020-01-07T02:21:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.