# Number of Shards for an Index **URL:** \ **Category:** Elasticsearch\ **Created:** [March 13, 2017, 8:25pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412 "2017-03-13T20:25:17Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![senthilvel](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@senthilvel](https://discuss.elastic.co/u/senthilvel)\ **Post date:** [March 13, 2017, 8:25pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412/1 "2017-03-13T20:25:17Z")
Hi, We are trying to come up with optimum number of shards for the Index. Based on [https://www.elastic.co/guide/en/elasticsearch/guide/current/capacity-planning.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/capacity-planning.html), we have done load testing and found that 750K documents per shard will be ideal. However, when we calculated the potential number of documents that could be indexed, we arrive at ~350 shards per index. Based on other internet searches, this seems to be high number. There are few pages suggested that ideal shard count would be 1 shard /node. We would have ~300M documents in the index with each document of \<1KB size. Could anyone please help with determining optimum number of shards/index? Thanks in advance. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 14, 2017, 6:35pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412/2 "2017-03-14T18:35:04Z")
Can you explain how you did your testing and arrived at the 750K number? ---
**Author:** ![senthilvel](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@senthilvel](https://discuss.elastic.co/u/senthilvel)\ **Post date:** [March 14, 2017, 6:39pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412/3 "2017-03-14T18:39:13Z")
Thanks for responding. My colleague had loaded the data in single node and single shard environment as suggested in the [https://www.elastic.co/guide/en/elasticsearch/guide/current/capacity-planning.html](https://www.elastic.co/guide/en/elasticsearch/guide/current/capacity-planning.html) , loaded real documents and ran the aggregation queries that we would run to see the response time. Based on the response time, we believe that 750K docs for a shard could be ideal number. please suggest if there are alternate ways. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [March 14, 2017, 6:59pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412/4 "2017-03-14T18:59:43Z")
Any more info you can share? What sort of agg, what sort of docs? How are you defining the SLA? ---
**Author:** ![senthilvel](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@senthilvel](https://discuss.elastic.co/u/senthilvel)\ **Post date:** [March 15, 2017, 3:21pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412/5 "2017-03-15T15:21:14Z")
HI Mark, Please find the details below. Below is the mapping. ``` PUT /weekly_performance { "settings": { "number_of_shards": 36, "number_of_replicas": 1, "index.routing.allocation.total_shards_per_node": 24 }, "mappings": { "weekly_performance": { "_all": { "enabled": false }, "properties": { "category1_id": { "type": "integer" }, "category2_id": { "type": "integer" }, "category3_id": { "type": "integer" }, "category2_name": { "type": "text", "index": "not_analyzed" }, "category4_id": { "type": "text", "index": "not_analyzed" }, "category5_id": { "type": "text", "index": "not_analyzed" }, "category6_id": { "type": "integer" }, "category6_alt_id": { "type": "text", "index": "not_analyzed" }, "location_id": { "type": "text", "index": "not_analyzed" }, "location_role": { "type": "text", "index": "not_analyzed" }, "location_channel": { "type": "text", "index": "not_analyzed" }, "actual_location_id": { "type": "text", "index": "not_analyzed" }, "actual_location_role": { "type": "text", "index": "not_analyzed" }, "actual_location_channel": { "type": "text", "index": "not_analyzed" }, "performance_code": { "type": "integer" }, "week_end_date": { "type": "date", "index": "not_analyzed" }, "week_id": { "type": "text", "index": "not_analyzed" }, "performance_unit_1": { "type": "double" }, "performance_a_amount_1": { "type": "double" }, "performance_b_amount_1": { "type": "double" }, "performance_unit_2": { "type": "double" }, "performance_a_amount_2": { "type": "double" }, "performance_b_amount_2": { "type": "double" }, "performance_unit_3": { "type": "double" }, "performance_a_amount_3": { "type": "double" }, "performance_b_amount_3": { "type": "double" }, "performance_unit_4": { "type": "double" }, "performance_a_amount_4": { "type": "double" }, "performance_b_amount_4": { "type": "double" }, "performance_unit_5": { "type": "double" }, "performance_a_amount_5": { "type": "double" }, "performance_b_amount_5": { "type": "double" }, "performance_unit_6": { "type": "double" }, "performance_a_amount_6": { "type": "double" }, "performance_b_amount_6": { "type": "double" }, "performance_unit_7": { "type": "double" }, "performance_a_amount_7": { "type": "double" }, "performance_unit_8": { "type": "double" }, "performance_a_amount_8": { "type": "double" }, "performance_b_amount_8": { "type": "double" }, "performance_unit_9": { "type": "double" }, "performance_a_amount_9": { "type": "double" }, "performance_b_amount_9": { "type": "double" }, "performance_unit_10": { "type": "double" }, "performance_a_amount_10": { "type": "double" }, "performance_unit_11": { "type": "double" }, "performance_a_amount_11": { "type": "double" }, "performance_unit_12": { "type": "double" }, "performance_a_amount_12": { "type": "double" }, "performance_unit_13": { "type": "double" }, "performance_a_amount_13": { "type": "double" }, "performance_a_amount_14": { "type": "double" }, "performance_a_amount_15": { "type": "double" }, "performance_unit_15": { "type": "double" }, "performance_a_amount_15": { "type": "double" } } } } } ``` ---
**Author:** ![senthilvel](https://avatars.discourse-cdn.com/v4/letter/s/e79b87/32.png) [@senthilvel](https://discuss.elastic.co/u/senthilvel)\ **Post date:** [March 15, 2017, 3:21pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412/6 "2017-03-15T15:21:29Z")
Below is the Aggregation that we use. ``` GET weekly_performance/_search { "size": 0, "query": { "bool": { "filter": [ { "range": { "week_end_date": { "format": "yyyy-MM-dd", "gte": "2017-01-28", "lte": "2017-02-04" } } }, { "term": { "category1_id": 99 } }, { "term": { "category2_id": 9 } } ] } }, "aggs": { "group_by_performance_code": { "terms": { "size": 10000, "field": "performance_code" }, "aggs": { "performance_unit_1": { "sum": { "field": "performance_unit_1" } }, "performance_a_amount_1" :{ "sum": { "field": "performance_a_amount_1" } }, "performance_b_amount_1" :{ "sum": { "field": "performance_b_amount_1" } }, "performance_unit_2" :{ "sum": { "field": "performance_unit_2" } }, "performance_a_amount_2" :{ "sum": { "field": "performance_a_amount_2" } }, "performance_b_amount_2" :{ "sum": { "field": "performance_b_amount_2" } }, "performance_unit_3" :{ "sum": { "field": "performance_unit_3" } }, "performance_a_amount_3" :{ "sum": { "field": "performance_a_amount_3" } }, "performance_b_amount_3" :{ "sum": { "field": "performance_b_amount_3" } }, "performance_unit_4" :{ "sum": { "field": "performance_unit_4" } }, "performance_a_amount_4" :{ "sum": { "field": "performance_a_amount_4" } }, "performance_b_amount_4" :{ "sum": { "field": "performance_b_amount_4" } }, "performance_unit_5" :{ "sum": { "field": "performance_unit_5" } }, "performance_a_amount_5" :{ "sum": { "field": "performance_a_amount_5" } }, "performance_b_amount_5" :{ "sum": { "field": "performance_b_amount_5" } }, "performance_a_amount_6" :{ "sum": { "field": "performance_a_amount_6" } }, "performance_b_amount_6" :{ "sum": { "field": "performance_b_amount_6" } }, "performance_unit_7" :{ "sum": { "field": "performance_unit_7" } }, "performance_a_amount_7" :{ "sum": { "field": "performance_a_amount_7" } }, "performance_unit_8" :{ "sum": { "field": "performance_unit_8" } }, "performance_a_amount_8" :{ "sum": { "field": "performance_a_amount_8" } }, "performance_b_amount_8" :{ "sum": { "field": "performance_b_amount_8" } }, "performance_unit_9" :{ "sum": { "field": "performance_unit_9" } }, "performance_a_amount_9" :{ "sum": { "field": "performance_a_amount_9" } }, "performance_b_amount_9" :{ "sum": { "field": "performance_b_amount_9" } }, "performance_b_amount_15" :{ "sum": { "field": "performance_b_amount_15" } }, "performance_a_amount_15" :{ "sum": { "field": "performance_a_amount_15" } }, "performance_unit_15" :{ "sum": { "field": "performance_unit_15" } }, "agg_source": { "top_hits": { "size": 1, "_source": { "includes": ["category1_id", "category2_id", "category3_id", "category2_name", "category4_id", "category5_id","category6_id", "performance_code"] } } } } } } } ``` We are looking at getting the results \< 1 second in most cases and \<2 second in worst case scenario. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 12, 2017, 3:21pm UTC](https://discuss.elastic.co/t/number-of-shards-for-an-index/78412/7 "2017-04-12T15:21:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.