# New indexes takes a long time to allocate its new shards

**URL:** https://discuss.elastic.co/t/new-indexes-takes-a-long-time-to-allocate-its-new-shards/32322
**Category:** Elasticsearch
**Created:** [October 16, 2015, 1:24am UTC](https://discuss.elastic.co/t/new-indexes-takes-a-long-time-to-allocate-its-new-shards/32322 "2015-10-16T01:24:04Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)
#### Post date: [October 16, 2015, 1:24am UTC](https://discuss.elastic.co/t/new-indexes-takes-a-long-time-to-allocate-its-new-shards/32322/1 "2015-10-16T01:24:04Z")

</div>

Hey Guys,

Got a question, the last 3 days (Under heavy sales volume) our index's via logstash rotate at GMT midnight.

We are noticing that it is taking +20 minutes for the new indexes to get allocated to a Node which during that time the Cluster is in a RED state

I have already adjusted my "Indexing" queue to  
indices.memory.index\_buffer\_size: 70%

because we are consuming about 200GB a day on this cluster and searches are comparatively light

What should I look for that would influence SHARE slowness in creation besides system load?

BTW, after the new indexes are create I have a total of ~1600 Shards (some indexes are small) maybe I should merge some of them and reduce the shard count instead of increasing # of nodes?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [October 16, 2015, 1:36am UTC](https://discuss.elastic.co/t/new-indexes-takes-a-long-time-to-allocate-its-new-shards/32322/2 "2015-10-16T01:36:45Z")

</div>

That's odd, new indices should be created very quickly as they have no data.

How many shards are you creating for the new indices, how many replicas?  
How big is your exisiting cluster state; how many existing indices, shards and replicas; how many nodes?

---

<div class="post-metadata">

### Author: ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)
#### Post date: [October 16, 2015, 2:25am UTC](https://discuss.elastic.co/t/new-indexes-takes-a-long-time-to-allocate-its-new-shards/32322/3 "2015-10-16T02:25:54Z")

</div>

Nuts just noticed some of the servers are GMT and some are EDT ick got to fix that

its 9 nodes - each one on its own system ( 8 cors 16GB ram 150GB for data disk )  
( at 8GB Heap, 150GB disk space each)  
100 index ( 5 shard with 1 replication) ,

SAR

```auto
02:12:01 AM CPU %user %nice %system %iowait %steal %idle
02:13:01 AM all 54.01 0.00 6.13 0.74 0.00 39.12
02:14:01 AM all 61.54 0.00 5.98 0.17 0.00 32.32
02:15:01 AM all 55.65 0.00 6.11 0.20 0.00 38.04
02:16:01 AM all 54.54 0.00 5.59 0.28 0.00 39.58
02:17:01 AM all 70.41 0.00 5.55 0.17 0.00 23.86
02:18:01 AM all 56.08 0.00 6.81 1.04 0.00 36.07
02:19:02 AM all 56.93 0.00 6.13 0.27 0.00 36.67
02:20:01 AM all 59.85 0.00 6.16 0.25 0.00 33.75
02:21:01 AM all 58.47 0.00 6.02 0.24 0.00 35.27
Average: all 43.77 0.00 5.61 0.40 0.00 50.22

```

---

<div class="post-metadata">

### Author: ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)
#### Post date: [October 16, 2015, 2:33am UTC](https://discuss.elastic.co/t/new-indexes-takes-a-long-time-to-allocate-its-new-shards/32322/4 "2015-10-16T02:33:00Z")

</div>

Oh and the HEAP is only 4GB out of the 8GB Total

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 5, 2017, 11:44pm UTC](https://discuss.elastic.co/t/new-indexes-takes-a-long-time-to-allocate-its-new-shards/32322/5 "2017-07-05T23:44:28Z")

</div>


