# Sharding question **URL:** \ **Category:** Elasticsearch\ **Created:** [May 30, 2017, 6:47pm UTC](https://discuss.elastic.co/t/sharding-question/87614 "2017-05-30T18:47:09Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![samdash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samdash/32/21628_2.png) [@samdash](https://discuss.elastic.co/u/samdash)\ **Post date:** [May 30, 2017, 6:47pm UTC](https://discuss.elastic.co/t/sharding-question/87614/1 "2017-05-30T18:47:09Z")
I have one index which has around 10K documents which may grow max 20K documents in few years,And i have another index which has more than 20Million documents and likely to grow more than double or triple in a year. how to pick my shards per index , this is for 3 node cluster all being master eligible [node.master: true and node.data:true] for 10K document shall i have 3 shards and 1 replica and for 20M documents index should i have 5 shards and 2 replica's ? If so , when i wanted to scale horizontally and add more nodes what will be the effect? and what should be the configuration if i wanted to add let's say 3 or 4 more nodes? Sam. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 31, 2017, 12:49am UTC](https://discuss.elastic.co/t/sharding-question/87614/2 "2017-05-31T00:49:39Z")
What sort of data is it? ---
**Author:** ![samdash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samdash/32/21628_2.png) [@samdash](https://discuss.elastic.co/u/samdash)\ **Post date:** [May 31, 2017, 1:52am UTC](https://discuss.elastic.co/t/sharding-question/87614/3 "2017-05-31T01:52:48Z")
20 Million Documents data is mostly user views/ratings and 10K document is just the tags for documents. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 31, 2017, 2:46am UTC](https://discuss.elastic.co/t/sharding-question/87614/4 "2017-05-31T02:46:23Z")
You should use time based indices for reviews, given they happen at a specific time. That means you can start with a daily/weekly/monthly index with a small shard count, and easily scale. ---
**Author:** ![samdash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samdash/32/21628_2.png) [@samdash](https://discuss.elastic.co/u/samdash)\ **Post date:** [May 31, 2017, 3:43am UTC](https://discuss.elastic.co/t/sharding-question/87614/5 "2017-05-31T03:43:24Z")
If using Time based index , i end up having 365 or 52 or 12 indexes , having 365 indexes is it a good thing? and if i have to search/aggregate using Transport Client using java for last 30 or 90 day's do i have to specific all the indexes while building the query? little bit newbie here , and how to create time based indexes? ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [May 31, 2017, 5:24am UTC](https://discuss.elastic.co/t/sharding-question/87614/6 "2017-05-31T05:24:51Z")
> [@samdash](#): > > If using Time based index , i end up having 365 or 52 or 12 indexes , having 365 indexes is it a good thing? Depends on your data volumes. I'd imagine for that 50 million size you can go with monthly with 1-3 primary shards and one replica. > [@samdash](#): > > do i have to specific all the indexes while building the query I'd use the java rest client if you are just starting. The transport one will be deprecated eventually. ---
**Author:** ![samdash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samdash/32/21628_2.png) [@samdash](https://discuss.elastic.co/u/samdash)\ **Post date:** [May 31, 2017, 4:02pm UTC](https://discuss.elastic.co/t/sharding-question/87614/7 "2017-05-31T16:02:11Z")
Does Rest Client has Aggregate Builders and all other features like pagination, Bulk Operations...etc that Transport client provides? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 31, 2017, 4:23pm UTC](https://discuss.elastic.co/t/sharding-question/87614/8 "2017-05-31T16:23:40Z")
Not yet. But this is coming... > ---
**Author:** ![samdash](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samdash/32/21628_2.png) [@samdash](https://discuss.elastic.co/u/samdash)\ **Post date:** [May 31, 2017, 4:54pm UTC](https://discuss.elastic.co/t/sharding-question/87614/9 "2017-05-31T16:54:18Z")
Thanks , we are going production soon and will be using Transport Client, In Which version are you going to deprecate Transport client? ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 31, 2017, 7:33pm UTC](https://discuss.elastic.co/t/sharding-question/87614/10 "2017-05-31T19:33:44Z")
I don't know. @javanna probably knows. ---
**Author:** ![javanna](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/javanna/32/4698_2.png) [@javanna](https://discuss.elastic.co/u/javanna)\ **Post date:** [May 31, 2017, 8:23pm UTC](https://discuss.elastic.co/t/sharding-question/87614/11 "2017-05-31T20:23:13Z")
Hi, we will most likely deprecate Transport Client in 6.0 , and remove it in 7.0 . ---
**Author:** ![Miguel1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miguel1/32/82094_2.png) [@Miguel1](https://discuss.elastic.co/u/Miguel1)\ **Post date:** [June 5, 2017, 6:58am UTC](https://discuss.elastic.co/t/sharding-question/87614/12 "2017-06-05T06:58:14Z")
Shards **shouldn't exceed 50 GB** (undersharding) and **shouldn't be less than 5 GB** (oversharding). In your case (considering that my average size for that amount of logs never exceeds 50 GB) I would go for 1 shards and 2 more replicas for taking advantage of the other two nodes. Remember that replicas are not backups, but they can provide you HA. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 3, 2017, 6:58am UTC](https://discuss.elastic.co/t/sharding-question/87614/13 "2017-07-03T06:58:56Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.