# Sharding a big index by name

**URL:** <https://discuss.elastic.co/t/sharding-a-big-index-by-name/5689>\
**Category:** Elasticsearch\
**Created:** [October 26, 2011, 12:53pm UTC](https://discuss.elastic.co/t/sharding-a-big-index-by-name/5689 "2011-10-26T12:53:06Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![David\_Stendardi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david_stendardi/32/2990_2.png) [@David\_Stendardi](https://discuss.elastic.co/u/David_Stendardi)\
**Post date:** [October 26, 2011, 12:53pm UTC](https://discuss.elastic.co/t/sharding-a-big-index-by-name/5689/1 "2011-10-26T12:53:06Z")

</div>

Hello !

I'am using elasticsearch intensively to index statistics. Since most of  
the requests will concern the last 2 months records and the volumetry will  
be quite important (400 000 records/day), i'am considering to add a year  
suffix to the index name, and use the template api.

example : curl -XPOST 'localhost:9200/statistic-2011/foo/' -d ...

Does it makes sense ? I'am wondering if this will optimize something or if  
elasticsearch already doing these kinds of optimizations internally ?

Cheers,

David Stendardi

---

<div class="post-metadata">

**Author:** ![David\_Stendardi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david_stendardi/32/2990_2.png) [@David\_Stendardi](https://discuss.elastic.co/u/David_Stendardi)\
**Post date:** [October 26, 2011, 12:55pm UTC](https://discuss.elastic.co/t/sharding-a-big-index-by-name/5689/2 "2011-10-26T12:55:00Z")

</div>

Addendum :  
It will probably be a Year - Month suffix rather than only Year. (ex :  
statistic-2011-10)

---

<div class="post-metadata">

**Author:** ![phobos182](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/phobos182/32/3011_2.png) [@phobos182](https://discuss.elastic.co/u/phobos182)\
**Post date:** [October 26, 2011, 1:10pm UTC](https://discuss.elastic.co/t/sharding-a-big-index-by-name/5689/3 "2011-10-26T13:10:25Z")

</div>

That's what we do at my company. We choose a index creation strategy to partition the data by time series. Then when we query it, it will look at a much smaller set of data rather than having it in one large index.

We choose a week index strategy (2011-42, 2011-43, 2011-44, ...) where the shard name is the year + week number.

Each index has 8 shards based on the size of the cluster. So when querying two weeks of information is hits 16 shards. When querying 3 weeks it hits 24 shards. Etc... Since ElasticSearch handles the parallel dispatch of requests, it's really not an issue to have high shard count if you have the machines to handle it.

When querying the data in ElasticSearch, you can choose what shards to execute the query on with a comma separated list.

Ex:

curl -XGET '[http://localhost:9200/2011-42,2011-43/\_search?q=user:kimchy](http://localhost:9200/2011-42,2011-43/_search?q=user:kimchy)'

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:50am UTC](https://discuss.elastic.co/t/sharding-a-big-index-by-name/5689/4 "2017-07-06T03:50:54Z")

</div>

Closed on July 6, 2017, 3:50am UTC