# Correct number of shards for 5.3 TB indices

**URL:** https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934
**Category:** Elasticsearch
**Created:** [April 19, 2017, 5:47pm UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934 "2017-04-19T17:47:13Z")
**Posts on this page:** 11
**Page:** 1

<div class="post-metadata">

### Author: ![mario12](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario12/32/29324_2.png) [@mario12](https://discuss.elastic.co/u/mario12)
#### Post date: [April 19, 2017, 5:47pm UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/1 "2017-04-19T17:47:13Z")

</div>

We have indices with total size of 5.3 TB. We have one replica for this indices. So, in total 10.6 TB. We are planning to start re-index process to make the shards smaller (currently 1TB per shard). We are planning to create new indices with 200 shards. This will make the shard size 27 GB (approximately). We have 33 node cluster and few small indices share this cluster. Please let me know if there is any disadvantages of having 200 shards.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 19, 2017, 11:58pm UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/2 "2017-04-19T23:58:26Z")

</div>

That's a reasonable size. Why is the index so large though?

---

<div class="post-metadata">

### Author: ![mario12](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario12/32/29324_2.png) [@mario12](https://discuss.elastic.co/u/mario12)
#### Post date: [April 20, 2017, 12:27am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/3 "2017-04-20T00:27:59Z")

</div>

Because, this indices is not time partitioned. We are planning to implement it in future.

I read that there is overhead in having more shards. Will having 200 shards (primary) + 200 (replica) for total no. of 33 nodes affect the performance negatively?

We are at 1.7 (java) and 1.7.2 (elasticsearch).

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 20, 2017, 12:31am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/4 "2017-04-20T00:31:08Z")

</div>

You'd have to test it to be sure. But given it's only 12 shards per node, I wouldn't imagine a big problem.

---

<div class="post-metadata">

### Author: ![mario12](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario12/32/29324_2.png) [@mario12](https://discuss.elastic.co/u/mario12)
#### Post date: [April 20, 2017, 12:35am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/5 "2017-04-20T00:35:58Z")

</div>

I will try to get this tested. What if we have 60 shards (p) + 60 shards (r)? This will reduce the no of shards per node ratio to 1:4. Is there any advantage of this proposal to 200 + 200 (we discussed it earlier)?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 20, 2017, 12:43am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/6 "2017-04-20T00:43:12Z")

</div>

Generally, more shards = higher indexing speeds, less shards = faster query speeds.

But it's all stuff you need to test as more/less shards is relative.

---

<div class="post-metadata">

### Author: ![mario12](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario12/32/29324_2.png) [@mario12](https://discuss.elastic.co/u/mario12)
#### Post date: [April 20, 2017, 12:46am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/7 "2017-04-20T00:46:59Z")

</div>

ok. Thanks for your help on this issue.

---

<div class="post-metadata">

### Author: ![mario12](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario12/32/29324_2.png) [@mario12](https://discuss.elastic.co/u/mario12)
#### Post date: [April 20, 2017, 12:48am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/8 "2017-04-20T00:48:26Z")

</div>

Is there a quick way find out if an indices is bound by read (query) or write (indexing)?

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [April 20, 2017, 1:18am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/9 "2017-04-20T01:18:21Z")

</div>

Use some kind of monitoring.

---

<div class="post-metadata">

### Author: ![mario12](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario12/32/29324_2.png) [@mario12](https://discuss.elastic.co/u/mario12)
#### Post date: [April 20, 2017, 1:51am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/10 "2017-04-20T01:51:31Z")

</div>

Thanks again.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 18, 2017, 1:53am UTC](https://discuss.elastic.co/t/correct-number-of-shards-for-5-3-tb-indices/82934/11 "2017-05-18T01:53:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
