# What will happen when shard size is too large **URL:** \ **Category:** Elasticsearch\ **Created:** [January 5, 2015, 2:13am UTC](https://discuss.elastic.co/t/what-will-happen-when-shard-size-is-too-large/21472 "2015-01-05T02:13:16Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Jinyuan\_Zhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinyuan_zhou/32/90185_2.png) [@Jinyuan\_Zhou](https://discuss.elastic.co/u/Jinyuan_Zhou)\ **Post date:** [January 5, 2015, 2:13am UTC](https://discuss.elastic.co/t/what-will-happen-when-shard-size-is-too-large/21472/1 "2015-01-05T02:13:16Z")
I read some where that max recommended shard size is about 50G. What are biggest bad consequences when shard size is too large. I have, for some indice, shard size more than 100G on a jvm with 29G fixed heap size. I suspect this is one factor that caused OutofMemory problem. However I don't know if large shard size will results in memory issue. I can only think of longer seek time. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [January 5, 2015, 3:53am UTC](https://discuss.elastic.co/t/what-will-happen-when-shard-size-is-too-large/21472/2 "2015-01-05T03:53:14Z")
Because you have to load data from the shard when you get a query, so the larger the shard the more data you load, and OOM or slower response times happen. It also helps recovery and reallocation if they are smaller. On 5 January 2015 at 13:13, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) wrote: > I read some where that max recommended shard size is about 50G. > What are biggest bad consequences when shard size is too large. > I have, for some indice, shard size more than 100G on a jvm with 29G > fixed heap size. I suspect this is one factor that caused OutofMemory > problem. > However I don't know if large shard size will results in memory issue. I > can only think of longer seek time. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-trGqU0u%3DCVcO8puRcDW-EcgdwLtGHd8EojgbHChVq2w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-trGqU0u%3DCVcO8puRcDW-EcgdwLtGHd8EojgbHChVq2w%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jinyuan\_Zhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinyuan_zhou/32/90185_2.png) [@Jinyuan\_Zhou](https://discuss.elastic.co/u/Jinyuan_Zhou)\ **Post date:** [January 5, 2015, 5:45am UTC](https://discuss.elastic.co/t/what-will-happen-when-shard-size-is-too-large/21472/3 "2015-01-05T05:45:48Z")
Thanks Mark, I understand the benefit for recovery and allocation. But could you help me a bit more about how much shards data needs to be loaded into heap at once when doing query? I thought it is the segments that need to be loaded/examined. If size of segment is limited and you load segment one at time (or you don't need load all segments at once ) in a shard, then it seems it is possible that large shard size may not have to increase the memory footprint. Regards, Jack Jinyuan (Jack) Zhou On Sun, Jan 4, 2015 at 7:53 PM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > Because you have to load data from the shard when you get a query, so the > larger the shard the more data you load, and OOM or slower response times > happen. > > It also helps recovery and reallocation if they are smaller. > > On 5 January 2015 at 13:13, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) wrote: > > > I read some where that max recommended shard size is about 50G. > > What are biggest bad consequences when shard size is too large. > > I have, for some indice, shard size more than 100G on a jvm with 29G > > fixed heap size. I suspect this is one factor that caused OutofMemory > > problem. > > However I don't know if large shard size will results in memory issue. I > > can only think of longer seek time. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/ce84aba7-67a5-4484-ac93-e86cc3f7384b%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/xzcSH6d0QY4/unsubscribe](https://groups.google.com/d/topic/elasticsearch/xzcSH6d0QY4/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-trGqU0u%3DCVcO8puRcDW-EcgdwLtGHd8EojgbHChVq2w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-trGqU0u%3DCVcO8puRcDW-EcgdwLtGHd8EojgbHChVq2w%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-trGqU0u%3DCVcO8puRcDW-EcgdwLtGHd8EojgbHChVq2w%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-trGqU0u%3DCVcO8puRcDW-EcgdwLtGHd8EojgbHChVq2w%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CANBTPCH0EfJBD0EtGz1Nd3k3tGQGUp%3DNfqu9xq5XNKj343Eoyw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CANBTPCH0EfJBD0EtGz1Nd3k3tGQGUp%3DNfqu9xq5XNKj343Eoyw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:40am UTC](https://discuss.elastic.co/t/what-will-happen-when-shard-size-is-too-large/21472/4 "2017-07-06T00:40:59Z")