# Daily indices for a period of 1 month with wildcard search taking long time

**URL:** https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470
**Category:** Kibana
**Created:** [July 27, 2016, 5:19am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470 "2016-07-27T05:19:33Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)
#### Post date: [July 27, 2016, 5:19am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/1 "2016-07-27T05:19:34Z")

</div>

Hi Team, i am having 1TB of data over the span of 1 month where we create daily indices and currently we are creating around 20 daily indices with total data of 30-35 GB per day . Field-stats api is taking 50 secs to get the 1000 shards for the period of 1 month and our backend store is Network Storage . Can you please let us know if field stats will take 50 secs in this scenario from Kibana.

---

<div class="post-metadata">

### Author: ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)
#### Post date: [July 27, 2016, 5:21am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/2 "2016-07-27T05:21:10Z")

</div>

Our Elasticsearch version is 2.1 and kibana is 4.3.0

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 27, 2016, 5:56am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/3 "2016-07-27T05:56:36Z")

</div>

Sounds like you have too many shards, that won't be helping.

---

<div class="post-metadata">

### Author: ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)
#### Post date: [July 27, 2016, 9:03am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/4 "2016-07-27T09:03:47Z")

</div>

Hi we are creating index per app and we have multiple apps. So instead of creating 20 primary shards per day , do you suggest us to consolidate the shards and use shard routing to route the small apps logs to 1 shard?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 27, 2016, 9:09pm UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/5 "2016-07-27T21:09:41Z")

</div>

Just create daily/weekly indices with a lot less shards

---

<div class="post-metadata">

### Author: ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)
#### Post date: [July 31, 2016, 1:52am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/6 "2016-07-31T01:52:15Z")

</div>

Also if 1 of my app has a need of 1 TB data per day , in that case i will have to create more number of shards, In that case i would not have solution to reduce number of shards . Please suggest.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 31, 2016, 2:14am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/7 "2016-07-31T02:14:37Z")

</div>

I missed that you have network storage, that will be slow.

You're best option is to remove the network storage and add more nodes.

---

<div class="post-metadata">

### Author: ![soodlikesjava](https://avatars.discourse-cdn.com/v4/letter/s/ee7513/32.png) [@soodlikesjava](https://discuss.elastic.co/u/soodlikesjava)
#### Post date: [July 31, 2016, 4:47am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/8 "2016-07-31T04:47:33Z")

</div>

Thanks for providing inputs . Can we achieve some sort of improvement by moving from NAS to SAN . Currently network storage is only option.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [July 31, 2016, 7:45am UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/9 "2016-07-31T07:45:17Z")

</div>

Maybe, you'd have to test.  
We don't usually recommend network based storage.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 1:43pm UTC](https://discuss.elastic.co/t/daily-indices-for-a-period-of-1-month-with-wildcard-search-taking-long-time/56470/10 "2017-07-06T13:43:06Z")

</div>


