# Wrong load distribution **URL:** \ **Category:** Elasticsearch\ **Created:** [March 31, 2015, 3:28pm UTC](https://discuss.elastic.co/t/wrong-load-distribution/23010 "2015-03-31T15:28:19Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Loic\_Wenkin](https://avatars.discourse-cdn.com/v4/letter/l/4491bb/32.png) [@Loic\_Wenkin](https://discuss.elastic.co/u/Loic_Wenkin)\ **Post date:** [March 31, 2015, 3:28pm UTC](https://discuss.elastic.co/t/wrong-load-distribution/23010/1 "2015-03-31T15:28:19Z")
Hi all, I meet a load distribution problem today and I browsed the Internet to find out someone having the same problem but unfortunately, no one seems to have. Here is my cluster configuration: - 4 machines with 16 Go Ram (9600 Mo allocated to JVM) - 8 core CPUs - Special cluster config: awareness of the vm host for shard primary and replica allocation (to avoid having both a primary and a replica on the same hardware). I tried a (huge) insert today, and it leads to one of the 4 nodes having a load average extremely higher than the three others (1.5 to 2 for the 3 "relax" servers vs. \>12 for the high loaded one). I was thinking that Elasticsearch was designed to avoid it. Am I wrong here? Any tips is welcome 🙂 Regards, Loïc -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 31, 2015, 10:02pm UTC](https://discuss.elastic.co/t/wrong-load-distribution/23010/2 "2015-03-31T22:02:59Z")
Do you have your shards equally distributed over the 4 nodes? Or do you use the default of 5 shards? Jörg On Tue, Mar 31, 2015 at 5:28 PM, Loïc Wenkin [loic.wenkin@gmail.com](mailto:loic.wenkin@gmail.com) wrote: > Hi all, > > I meet a load distribution problem today and I browsed the Internet to > find out someone having the same problem but unfortunately, no one seems to > have. Here is my cluster configuration: > > - 4 machines with 16 Go Ram (9600 Mo allocated to JVM) > - 8 core CPUs > - Special cluster config: awareness of the vm host for shard primary and > replica allocation (to avoid having both a primary and a replica on the > same hardware). > > I tried a (huge) insert today, and it leads to one of the 4 nodes having a > load average extremely higher than the three others (1.5 to 2 for the 3 > "relax" servers vs. \>12 for the high loaded one). I was thinking that > Elasticsearch was designed to avoid it. Am I wrong here? > > Any tips is welcome 🙂 > > Regards, > Loïc > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH-iy-%3D5NPb0YoYuydwd-MXR%2BuLQ7p\_4m9mNVrNLyEPyg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH-iy-%3D5NPb0YoYuydwd-MXR%2BuLQ7p_4m9mNVrNLyEPyg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Loic\_Wenkin](https://avatars.discourse-cdn.com/v4/letter/l/4491bb/32.png) [@Loic\_Wenkin](https://discuss.elastic.co/u/Loic_Wenkin)\ **Post date:** [April 1, 2015, 7:21am UTC](https://discuss.elastic.co/t/wrong-load-distribution/23010/3 "2015-04-01T07:21:13Z")
Hi Jörg, Thanks for your reply. I have the default 5 shards, but I have multiple indexes, and all nodes have about the same number of shards on them. Regards, Loïc Le mercredi 1 avril 2015 00:03:06 UTC+2, Jörg Prante a écrit : > Do you have your shards equally distributed over the 4 nodes? Or do you > use the default of 5 shards? > > Jörg > > On Tue, Mar 31, 2015 at 5:28 PM, Loïc Wenkin \<[loic....@gmail.com](mailto:loic....@gmail.com) > \\> wrote: > > > Hi all, > > > > I meet a load distribution problem today and I browsed the Internet to > > find out someone having the same problem but unfortunately, no one seems to > > have. Here is my cluster configuration: > > > > - 4 machines with 16 Go Ram (9600 Mo allocated to JVM) > > - 8 core CPUs > > - Special cluster config: awareness of the vm host for shard primary and > > replica allocation (to avoid having both a primary and a replica on the > > same hardware). > > > > I tried a (huge) insert today, and it leads to one of the 4 nodes having > > a load average extremely higher than the three others (1.5 to 2 for the 3 > > "relax" servers vs. \>12 for the high loaded one). I was thinking that > > Elasticsearch was designed to avoid it. Am I wrong here? > > > > Any tips is welcome 🙂 > > > > Regards, > > Loïc > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/4abd9e01-b4b3-4b5c-9ccf-66853c84b255%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4abd9e01-b4b3-4b5c-9ccf-66853c84b255%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [April 1, 2015, 8:15am UTC](https://discuss.elastic.co/t/wrong-load-distribution/23010/4 "2015-04-01T08:15:07Z")
You should set k\*n shards for your indexes to avoid balancing troubles with k being a constant integer and n being the number of nodes. Also keep an eye on dynamic mapping. if you have add thousands of new field names steadily over time, you exercise the master node. Jörg On Wed, Apr 1, 2015 at 9:21 AM, Loïc Wenkin [loic.wenkin@gmail.com](mailto:loic.wenkin@gmail.com) wrote: > Hi Jörg, > > Thanks for your reply. I have the default 5 shards, but I have multiple > indexes, and all nodes have about the same number of shards on them. > > Regards, > Loïc > > Le mercredi 1 avril 2015 00:03:06 UTC+2, Jörg Prante a écrit : > > > Do you have your shards equally distributed over the 4 nodes? Or do you > > use the default of 5 shards? > > > > Jörg > > > > On Tue, Mar 31, 2015 at 5:28 PM, Loïc Wenkin [loic....@gmail.com](mailto:loic....@gmail.com) wrote: > > > > > Hi all, > > > > > > I meet a load distribution problem today and I browsed the Internet to > > > find out someone having the same problem but unfortunately, no one seems to > > > have. Here is my cluster configuration: > > > > > > - 4 machines with 16 Go Ram (9600 Mo allocated to JVM) > > > - 8 core CPUs > > > - Special cluster config: awareness of the vm host for shard primary and > > > replica allocation (to avoid having both a primary and a replica on the > > > same hardware). > > > > > > I tried a (huge) insert today, and it leads to one of the 4 nodes having > > > a load average extremely higher than the three others (1.5 to 2 for the 3 > > > "relax" servers vs. \>12 for the high loaded one). I was thinking that > > > Elasticsearch was designed to avoid it. Am I wrong here? > > > > > > Any tips is welcome 🙂 > > > > > > Regards, > > > Loïc > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/d1aa9ed2-cdeb-4e07-aa7f-b760fb5f728e%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/4abd9e01-b4b3-4b5c-9ccf-66853c84b255%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/4abd9e01-b4b3-4b5c-9ccf-66853c84b255%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/4abd9e01-b4b3-4b5c-9ccf-66853c84b255%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/4abd9e01-b4b3-4b5c-9ccf-66853c84b255%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGdqp5jnzJXis6J82fN%2BwRaJAcAWUpQr7OFi24cVQwgLQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGdqp5jnzJXis6J82fN%2BwRaJAcAWUpQr7OFi24cVQwgLQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:22am UTC](https://discuss.elastic.co/t/wrong-load-distribution/23010/5 "2017-07-06T00:22:26Z")