# Elasticserch 2.3.3 Replicas keep getting marked as failed in a 4 node cluster

**URL:** <https://discuss.elastic.co/t/elasticserch-2-3-3-replicas-keep-getting-marked-as-failed-in-a-4-node-cluster/63137>\
**Category:** Elasticsearch\
**Created:** [October 16, 2016, 6:32pm UTC](https://discuss.elastic.co/t/elasticserch-2-3-3-replicas-keep-getting-marked-as-failed-in-a-4-node-cluster/63137 "2016-10-16T18:32:07Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![S\_A\_M](https://avatars.discourse-cdn.com/v4/letter/s/e0b2c6/32.png) [@S\_A\_M](https://discuss.elastic.co/u/S_A_M)\
**Post date:** [October 16, 2016, 6:32pm UTC](https://discuss.elastic.co/t/elasticserch-2-3-3-replicas-keep-getting-marked-as-failed-in-a-4-node-cluster/63137/1 "2016-10-16T18:32:08Z")

</div>

Any ideas what the issue may be? any assistance would be appreciated.

We're using Elasticsearch 2.3.3 currently and are seeing lots of these somewhat frequently in the logs - not really sure why.:

## [2016-10-16 03:22:27,922][WARN][indices.cluster] [data1] [[idx-1955361900][7]] marking and sending shard failed due to [engine failure, reason [indices:admin/flush[s] failed on replica]] [idx-1955361900][[idx-1955361900][7]] FlushNotAllowedEngineException[already flushing...]

[2016-10-16 03:22:40,792][WARN][indices.cluster] [data1] [[idx-1955361900][5]] marking and sending shard failed due to [engine failure, reason [indices:admin/flush[s] failed on replica]]  
[idx-1955361900][[idx-1955361900][5]] FlushNotAllowedEngineException[already flushing...]

This is also a fairly new cluster implementation so this issue did not just appear.

Cluster info:  
{  
"cluster\_name" : "elasticsearch",  
"status" : "green",  
"timed\_out" : false,  
"number\_of\_nodes" : 5,  
"number\_of\_data\_nodes" : 4,  
"active\_primary\_shards" : 1104,  
"active\_shards" : 2046,  
"relocating\_shards" : 0,  
"initializing\_shards" : 0,  
"unassigned\_shards" : 0,  
"delayed\_unassigned\_shards" : 0,  
"number\_of\_pending\_tasks" : 0,  
"number\_of\_in\_flight\_fetch" : 0,  
"task\_max\_waiting\_in\_queue\_millis" : 0,  
"active\_shards\_percent\_as\_number" : 100.0  
}

---

<div class="post-metadata">

**Author:** ![Bruce\_Ritchie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bruce_ritchie/32/9370_2.png) [@Bruce\_Ritchie](https://discuss.elastic.co/u/Bruce_Ritchie)\
**Post date:** [October 17, 2016, 2:51pm UTC](https://discuss.elastic.co/t/elasticserch-2-3-3-replicas-keep-getting-marked-as-failed-in-a-4-node-cluster/63137/2 "2016-10-17T14:51:16Z")

</div>

Looks to be a bug fixed in 2.4.1 - [https://github.com/elastic/elasticsearch/pull/20632](https://github.com/elastic/elasticsearch/pull/20632)

---

<div class="post-metadata">

**Author:** ![S\_A\_M](https://avatars.discourse-cdn.com/v4/letter/s/e0b2c6/32.png) [@S\_A\_M](https://discuss.elastic.co/u/S_A_M)\
**Post date:** [October 17, 2016, 3:08pm UTC](https://discuss.elastic.co/t/elasticserch-2-3-3-replicas-keep-getting-marked-as-failed-in-a-4-node-cluster/63137/3 "2016-10-17T15:08:03Z")

</div>

Thanks!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:11pm UTC](https://discuss.elastic.co/t/elasticserch-2-3-3-replicas-keep-getting-marked-as-failed-in-a-4-node-cluster/63137/4 "2017-07-05T22:11:43Z")

</div>