# Shards throughput for a big index?

**URL:** <https://discuss.elastic.co/t/shards-throughput-for-a-big-index/210134>\
**Category:** Elasticsearch\
**Created:** [December 2, 2019, 8:18am UTC](https://discuss.elastic.co/t/shards-throughput-for-a-big-index/210134 "2019-12-02T08:18:59Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [December 2, 2019, 8:18am UTC](https://discuss.elastic.co/t/shards-throughput-for-a-big-index/210134/1 "2019-12-02T08:18:59Z")

</div>

Hi elastic family,

Before I ask, let me elaborate my scenario.  
I centralize the logs to elasticsearch using logstash.  
And I use monthly indices format.  
Well it's been working properly.

As far as I know, limit of a shard is 50G.  
At the moment, I use default shards throughput (5-shard in primary, 5-shard in secondary) for an index.  
I read one post that 35 GB is good for each shard.  
So I assume 200 GB is suitable for my monthly index.

My concern is that if logs become bigger then 200 GB a month, could be 300 GB for example.  
Do I need to increase shards throughput like (7-shard in primary, 7-shard in secondary)?  
What should I do? What is the best practice for such scenario?

Looking forward to replying...  
Thanks so much

---

<div class="post-metadata">

**Author:** ![DavidTurner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/davidturner/32/22453_2.png) [@DavidTurner](https://discuss.elastic.co/u/DavidTurner)\
**Post date:** [December 2, 2019, 11:22am UTC](https://discuss.elastic.co/t/shards-throughput-for-a-big-index/210134/2 "2019-12-02T11:22:49Z")

</div>

> [@shwesinhan](#):
>
> As far as I know, limit of a shard is 50G.

This is not true, there is no such limit on the size of a shard. It might be a good idea not to let your shards grow too large since larger shards take proportionally longer to recover, and 50GB is a reasonable target size, but it is not a limit.

> [@shwesinhan](#):
>
> What should I do? What is the best practice for such scenario?

Use as few primaries as possible (1 is the default in recent versions) and use [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) to rollover to a new index when it reaches the right size or age.

---

<div class="post-metadata">

**Author:** ![shwesinhan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shwesinhan/32/78663_2.png) [@shwesinhan](https://discuss.elastic.co/u/shwesinhan)\
**Post date:** [December 3, 2019, 7:15am UTC](https://discuss.elastic.co/t/shards-throughput-for-a-big-index/210134/3 "2019-12-03T07:15:15Z")

</div>

Thank you @DavidTurner for correcting my wrong knowledge, and advice.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 31, 2019, 7:15am UTC](https://discuss.elastic.co/t/shards-throughput-for-a-big-index/210134/4 "2019-12-31T07:15:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
