# Number of shards

**URL:** <https://discuss.elastic.co/t/number-of-shards/26481>\
**Category:** Elasticsearch\
**Created:** [July 29, 2015, 12:07pm UTC](https://discuss.elastic.co/t/number-of-shards/26481 "2015-07-29T12:07:59Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![saif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saif/32/37431_2.png) [@saif](https://discuss.elastic.co/u/saif)\
**Post date:** [July 29, 2015, 12:07pm UTC](https://discuss.elastic.co/t/number-of-shards/26481/1 "2015-07-29T12:07:59Z")

</div>

Hello  
i have an ES cluster run with 3 nodes "3 servers"  
2 master and 3 data nodes

rsyslog-2015.07.28  
size: 4.72Gi (7.55Gi)  
docs: 30,477,510 (30,477,510)

i want to optimize the number of shards pr node  
actually 1 replica and 5 shards

i think to use 6 shards  
node 1: 0+3+R5+R2  
node 2: 1+4+R4+R1  
node 3: 2+5+R3+R0

so what is the best number of shards for my cluster ? 6 ?  
thank you

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 29, 2015, 10:27pm UTC](https://discuss.elastic.co/t/number-of-shards/26481/2 "2015-07-29T22:27:38Z")

</div>

You have 3 data nodes, right? The way you have put it is a little confusing.

---

<div class="post-metadata">

**Author:** ![saif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saif/32/37431_2.png) [@saif](https://discuss.elastic.co/u/saif)\
**Post date:** [July 30, 2015, 8:08am UTC](https://discuss.elastic.co/t/number-of-shards/26481/3 "2015-07-30T08:08:20Z")

</div>

hi  
yes 3 nodes  
6 shared 1 replica  
every node hosted in

Architecture: x86\_64  
CPU op-mode(s): 32-bit, 64-bit  
Byte Order: Little Endian  
CPU(s): 8  
On-line CPU(s) list: 0-7  
Thread(s) per core: 1  
Core(s) per socket: 4  
Socket(s): 2  
NUMA node(s): 1  
Vendor ID: GenuineIntel  
CPU family: 6  
Model: 23  
Stepping: 6  
CPU MHz: 2499.861  
BogoMIPS: 4999.29  
Virtualization: VT-x  
L1d cache: 32K  
L1i cache: 32K  
L2 cache: 6144K  
NUMA node0 CPU(s): 0-7

and 4TB of storage Raid 0 and 8G of ram

its ok ?

in head plugin  
node 1 : (2) 3 4 (5)  
node 2 : (0) 1 2 (3)  
node 1 : 0 (1) (4) 5

you think that ok for 30,477,510 docs per day ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 30, 2015, 10:24am UTC](https://discuss.elastic.co/t/number-of-shards/26481/4 "2015-07-30T10:24:18Z")

</div>

I think that'll be ok, you could even reduce that to 3 shards easily, it's not a lot of data.

---

<div class="post-metadata">

**Author:** ![saif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saif/32/37431_2.png) [@saif](https://discuss.elastic.co/u/saif)\
**Post date:** [July 30, 2015, 11:09am UTC](https://discuss.elastic.co/t/number-of-shards/26481/5 "2015-07-30T11:09:21Z")

</div>

hi  
i have 30 million docs per day  
i think 3 shared can't support that number of document

anyone know how to calculate shards number ?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 30, 2015, 11:59am UTC](https://discuss.elastic.co/t/number-of-shards/26481/6 "2015-07-30T11:59:15Z")

</div>

30 million docs is not a lot, 3 shards can deal with that very easily.

---

<div class="post-metadata">

**Author:** ![terrasacer](https://avatars.discourse-cdn.com/v4/letter/t/ecd19e/32.png) [@terrasacer](https://discuss.elastic.co/u/terrasacer)\
**Post date:** [July 30, 2015, 12:16pm UTC](https://discuss.elastic.co/t/number-of-shards/26481/7 "2015-07-30T12:16:52Z")

</div>

if there the 5 shards of 3 data node 2 replica unnecessary because all of will be not allocated.

---

<div class="post-metadata">

**Author:** ![saif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saif/32/37431_2.png) [@saif](https://discuss.elastic.co/u/saif)\
**Post date:** [July 30, 2015, 12:26pm UTC](https://discuss.elastic.co/t/number-of-shards/26481/8 "2015-07-30T12:26:51Z")

</div>

30 million per day  
and after 12 months  
365 \* 30 million  
?....

---

<div class="post-metadata">

**Author:** ![saif](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/saif/32/37431_2.png) [@saif](https://discuss.elastic.co/u/saif)\
**Post date:** [July 30, 2015, 12:31pm UTC](https://discuss.elastic.co/t/number-of-shards/26481/9 "2015-07-30T12:31:37Z")

</div>

sorry yes 1 replica

---

<div class="post-metadata">

**Author:** ![rodrigo](https://avatars.discourse-cdn.com/v4/letter/r/4af34b/32.png) [@rodrigo](https://discuss.elastic.co/u/rodrigo)\
**Post date:** [July 30, 2015, 7:40pm UTC](https://discuss.elastic.co/t/number-of-shards/26481/10 "2015-07-30T19:40:05Z")

</div>

"Enough" is a very subjective concept...What are your expectations ? 30 mi docs/day are also subjective (in general a breeze for ES to handle), what is in your document ? (to the extreme, it can contain just an id or a full book or logdump, etc). Are you trying to optimize search speed (do you have an SLA) ? Are you worried about scaling up ? Are you keeping the docs forever, using it for storage ? I think if you have better answers for this you can get better responses...if you have daily data, you can use a daily index (foo\_20150730, foo\_20150801) and an alias to add flexibility to your setup. Number of shards, hosts is always subjective to your use case. Number of replicas attached to your failure tolerance.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 11:58pm UTC](https://discuss.elastic.co/t/number-of-shards/26481/11 "2017-07-05T23:58:05Z")

</div>


