# Replica shards stuck in Initialization phase

**URL:** <https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448>\
**Category:** Elasticsearch\
**Created:** [April 26, 2016, 3:25pm UTC](https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448 "2016-04-26T15:25:25Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![vb3](https://avatars.discourse-cdn.com/v4/letter/v/7993a0/32.png) [@vb3](https://discuss.elastic.co/u/vb3)\
**Post date:** [April 26, 2016, 3:25pm UTC](https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448/1 "2016-04-26T15:25:25Z")

</div>

Hi all,

I just did a full cluster restart, and when the nodes came up, none of the replica shards were getting initialized.  
I could make the cluster go green, by setting all the nodes to 0 replicas, but when changed to 1 or 2 replicas, the replicas are stuck in initializing phase (yellow cluster).

- I have tried restarting cluster twice already
- There are no .recovery files in the translog
- I can continue using cluster with no replicas

Using ES 1.7.1 and JVM 1.8.0\_92

Can anybody help with this issue?

EDIT: finding this in the logs:

```auto
[WARN] (elasticsearch[][generic][T#115]) org.elasticsearch.indices.cluster: [] [[index-2016.04][1]] marking and sending shard failed due to [failed recovery]
org.elasticsearch.indices.recovery.RecoveryFailedException: [index-2016.04][1]: Recovery failed from [][][][inet[/]]{max_local_storage_nodes=1, AZ=us-west-2a, master=false} into [][][][inet[/]]{max_local_storage_nodes=1, AZ=us-west-2c, master=false} (no activity after [30m])
        at org.elasticsearch.indices.recovery.RecoveriesCollection$RecoveryMonitor.doRun(RecoveriesCollection.java:235)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:36)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.elasticsearch.ElasticsearchTimeoutException: no activity after [30m]
        ... 5 more

```

Thanks,

V

---

<div class="post-metadata">

**Author:** ![vb3](https://avatars.discourse-cdn.com/v4/letter/v/7993a0/32.png) [@vb3](https://discuss.elastic.co/u/vb3)\
**Post date:** [April 26, 2016, 8:21pm UTC](https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448/2 "2016-04-26T20:21:28Z")

</div>

Fixed by following: [http://stackoverflow.com/a/33026075](http://stackoverflow.com/a/33026075)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 27, 2016, 3:45am UTC](https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448/3 "2016-04-27T03:45:52Z")

</div>

That stackoverflow post is totally wrong, KB will not stop any sort of shard action from happening.

And the solution is a _really_ bad idea. **Really bad**.  
Deleting the segments gen files basically stops Lucene/ES from keeping track of deleted files.

---

<div class="post-metadata">

**Author:** ![vb3](https://avatars.discourse-cdn.com/v4/letter/v/7993a0/32.png) [@vb3](https://discuss.elastic.co/u/vb3)\
**Post date:** [April 27, 2016, 2:32pm UTC](https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448/4 "2016-04-27T14:32:12Z")

</div>

KB? Kibana? I know our issue was not created by Kibana, but it seems to have gone away when I removed the segments gen files. and restarted the cluster.  
But if that wasnt the issue, @warkolm could you help diagnose the issue? What could be the reason for the replicas from not being initialized?

thanks!

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 28, 2016, 6:59am UTC](https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448/5 "2016-04-28T06:59:37Z")

</div>

You'd need to turn the logging level up when it happens again to get more info.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:55pm UTC](https://discuss.elastic.co/t/replica-shards-stuck-in-initialization-phase/48448/6 "2017-07-05T22:55:40Z")

</div>