# Shard constantly initializing after a bulk index (aka the Shard Dance)

**URL:** <https://discuss.elastic.co/t/shard-constantly-initializing-after-a-bulk-index-aka-the-shard-dance/7618>\
**Category:** Elasticsearch\
**Created:** [May 9, 2012, 4:30pm UTC](https://discuss.elastic.co/t/shard-constantly-initializing-after-a-bulk-index-aka-the-shard-dance/7618 "2012-05-09T16:30:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [May 9, 2012, 4:30pm UTC](https://discuss.elastic.co/t/shard-constantly-initializing-after-a-bulk-index-aka-the-shard-dance/7618/1 "2012-05-09T16:30:29Z")

</div>

Bulk loaded 28m documents to a new index with 0 replicas and with  
refresh disabled using ES 0.19.2. After the initial import, the # of  
replicas was set to 1 and reset the refresh interval to 1 second. Two  
shards have been moving between 3 (of the 4) different nodes for  
almost 12 hours.

Some stats:

{  
cluster\_name: ESCluster  
status: yellow  
timed\_out: false  
number\_of\_nodes: 4  
number\_of\_data\_nodes: 4  
active\_primary\_shards: 5  
active\_shards: 8  
relocating\_shards: 0  
initializing\_shards: 2  
unassigned\_shards: 0  
}

Sample visualization of two clusters states, taken seconds apart:  
[![](http://i.imgur.com/removed.png) ](http://i.imgur.com/removed.png)

None of the nodes were restarted after the import. Other possible  
relevant settings:

gateway.expected\_nodes: 1  
discovery.zen.ping.multicast.enabled: false  
discovery.zen.ping.unicast.hosts: ["192.168.50.101", "192.168.50.102"]

Note that the unicast hosts only contain 2 out of the 4 hosts. Seems  
to have worked well so far. The cluster is in development and none of  
the settings (number of shards/replicas/nodes) have been tuned.  
Working primarily on the import/search flow.

Cheers,

Ivan

---

<div class="post-metadata">

**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\
**Post date:** [May 9, 2012, 6:10pm UTC](https://discuss.elastic.co/t/shard-constantly-initializing-after-a-bulk-index-aka-the-shard-dance/7618/2 "2012-05-09T18:10:15Z")

</div>

Sorry, it was user error in the end. Ran out of disk space. Moved logs  
into a new location and was tailing the old ones.

--  
Ivan

On Wed, May 9, 2012 at 9:30 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote:

> Bulk loaded 28m documents to a new index with 0 replicas and with  
> refresh disabled using ES 0.19.2. After the initial import, the # of  
> replicas was set to 1 and reset the refresh interval to 1 second. Two  
> shards have been moving between 3 (of the 4) different nodes for  
> almost 12 hours.
> 
> Some stats:
> 
> {  
> cluster\_name: ESCluster  
> status: yellow  
> timed\_out: false  
> number\_of\_nodes: 4  
> number\_of\_data\_nodes: 4  
> active\_primary\_shards: 5  
> active\_shards: 8  
> relocating\_shards: 0  
> initializing\_shards: 2  
> unassigned\_shards: 0  
> }
> 
> Sample visualization of two clusters states, taken seconds apart:  
> [http://i.imgur.com/bRgpZ.png](http://i.imgur.com/bRgpZ.png)
> 
> None of the nodes were restarted after the import. Other possible  
> relevant settings:
> 
> gateway.expected\_nodes: 1  
> discovery.zen.ping.multicast.enabled: false  
> discovery.zen.ping.unicast.hosts: ["192.168.50.101", "192.168.50.102"]
> 
> Note that the unicast hosts only contain 2 out of the 4 hosts. Seems  
> to have worked well so far. The cluster is in development and none of  
> the settings (number of shards/replicas/nodes) have been tuned.  
> Working primarily on the import/search flow.
> 
> Cheers,
> 
> Ivan

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:29am UTC](https://discuss.elastic.co/t/shard-constantly-initializing-after-a-bulk-index-aka-the-shard-dance/7618/3 "2017-07-06T03:29:34Z")

</div>