# Should ES\_HEAP\_SIZE be less than 31G? **URL:** \ **Category:** Elasticsearch\ **Created:** [September 3, 2014, 4:34pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595 "2014-09-03T16:34:10Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![Jinyuan\_Zhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinyuan_zhou/32/90185_2.png) [@Jinyuan\_Zhou](https://discuss.elastic.co/u/Jinyuan_Zhou)\ **Post date:** [September 3, 2014, 4:34pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/1 "2014-09-03T16:34:10Z")
I read somewhere that the ES\_HEAP\_SIZE is best to be less than 31G, in this case JVM can use an 32 bit number to address memory locations. If my server have about 64G ram. This seems a perfect. But what If I have a server with 128G RAM and sufficient CPU's. Which configuration is better a) two nodes on server each with ES\_HEAP\_SIZE say about 31G. b) one node on the server with ES\_HEAP\_SIZE say 64G. Thanks, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![vineeth\_mohan\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vineeth_mohan_2/32/747_2.png) [@vineeth\_mohan\_2](https://discuss.elastic.co/u/vineeth_mohan_2)\ **Post date:** [September 3, 2014, 4:40pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/2 "2014-09-03T16:40:21Z")
Hi , I am not sure if there is anything related to ES that makes a high RAM problematic , but in general very high memory is not advised in java systems. One of the reasons being the cobra effect which stems from the fact that garbage collection might need more time for higher memory. COBRA EFFECT - [https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect](https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect) Thanks Vineeth On Wed, Sep 3, 2014 at 10:04 PM, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) wrote: > I read somewhere that the ES\_HEAP\_SIZE is best to be less than 31G, in > this case JVM can use an 32 bit number to address memory locations. If > my server have about 64G ram. This seems a perfect. But what If I have a > server with 128G RAM and sufficient CPU's. Which configuration is better > a) two nodes on server each with ES\_HEAP\_SIZE say about 31G. b) one node > on the server with ES\_HEAP\_SIZE say 64G. > Thanks, > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [September 3, 2014, 4:50pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/3 "2014-09-03T16:50:12Z")
Yeah. I'd prefer 2 nodes on the same machine but using that: [http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-awareness](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html#allocation-awareness) So primaries and replicas are not allocated on the same machine. You could also add a 3rd node (master only - 512Mb/1Gb max) Best -- David Pilato | Technical Advocate | [Elasticsearch.com](http://Elasticsearch.com) @dadoonet | @elasticsearchfr Le 3 septembre 2014 à 18:40:27, vineeth mohan ([vm.vineethmohan@gmail.com](mailto:vm.vineethmohan@gmail.com)) a écrit: Hi , I am not sure if there is anything related to ES that makes a high RAM problematic , but in general very high memory is not advised in java systems. One of the reasons being the cobra effect which stems from the fact that garbage collection might need more time for higher memory. COBRA EFFECT - [https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect](https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect) Thanks Vineeth On Wed, Sep 3, 2014 at 10:04 PM, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) wrote: I read somewhere that the ES\_HEAP\_SIZE is best to be less than 31G, in this case JVM can use an 32 bit number to address memory locations. If my server have about 64G ram. This seems a perfect. But what If I have a server with 128G RAM and sufficient CPU's. Which configuration is better a) two nodes on server each with ES\_HEAP\_SIZE say about 31G. b) one node on the server with ES\_HEAP\_SIZE say 64G. Thanks, -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/etPan.540746c4.333ab105.ec11%40MacBook-Air-de-David.local](https://groups.google.com/d/msgid/elasticsearch/etPan.540746c4.333ab105.ec11%40MacBook-Air-de-David.local). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [September 3, 2014, 5:02pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/4 "2014-09-03T17:02:05Z")
The actual limitation in Java is compressed pointers: [http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html#compressedOop](http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html#compressedOop) Thankfully Elasticsearch can run multiple nodes on the same server. Just pay attention to the direct size (off heap memory), mlockall settings and Lucene merge settings, as well as the allocation property referenced by David. Elasticsearch scales quite nicely vertically, so I would explore that option if possible. Cheers, Ivan On Wed, Sep 3, 2014 at 9:40 AM, vineeth mohan [vm.vineethmohan@gmail.com](mailto:vm.vineethmohan@gmail.com) wrote: > Hi , > > I am not sure if there is anything related to ES that makes a high RAM > problematic , but in general very high memory is not advised in java > systems. One of the reasons being the cobra effect which stems from the > fact that garbage collection might need more time for higher memory. > > COBRA EFFECT - > [https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect](https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect) > > Thanks > Vineeth > > On Wed, Sep 3, 2014 at 10:04 PM, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) > wrote: > > > I read somewhere that the ES\_HEAP\_SIZE is best to be less than 31G, in > > this case JVM can use an 32 bit number to address memory locations. If > > my server have about 64G ram. This seems a perfect. But what If I have a > > server with 128G RAM and sufficient CPU's. Which configuration is better > > a) two nodes on server each with ES\_HEAP\_SIZE say about 31G. b) one node > > on the server with ES\_HEAP\_SIZE say 64G. > > Thanks, > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jinyuan\_Zhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinyuan_zhou/32/90185_2.png) [@Jinyuan\_Zhou](https://discuss.elastic.co/u/Jinyuan_Zhou)\ **Post date:** [September 3, 2014, 5:43pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/5 "2014-09-03T17:43:00Z")
Thanks for answering my questions. I also followed all these interesting reading and learned a lot. Cheers, Jack Jinyuan (Jack) Zhou On Wed, Sep 3, 2014 at 10:02 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > The actual limitation in Java is compressed pointers: > > [Java ™ HotSpot Virtual Machine Performance Enhancements](http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html#compressedOop) > > Thankfully Elasticsearch can run multiple nodes on the same server. Just > pay attention to the direct size (off heap memory), mlockall settings and > Lucene merge settings, as well as the allocation property referenced by > David. Elasticsearch scales quite nicely vertically, so I would explore > that option if possible. > > Cheers, > > Ivan > > On Wed, Sep 3, 2014 at 9:40 AM, vineeth mohan [vm.vineethmohan@gmail.com](mailto:vm.vineethmohan@gmail.com) > wrote: > > > Hi , > > > > I am not sure if there is anything related to ES that makes a high RAM > > problematic , but in general very high memory is not advised in java > > systems. One of the reasons being the cobra effect which stems from the > > fact that garbage collection might need more time for higher memory. > > > > COBRA EFFECT - > > [https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect](https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect) > > > > Thanks > > Vineeth > > > > On Wed, Sep 3, 2014 at 10:04 PM, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) > > wrote: > > > > > I read somewhere that the ES\_HEAP\_SIZE is best to be less than 31G, in > > > this case JVM can use an 32 bit number to address memory locations. If > > > my server have about 64G ram. This seems a perfect. But what If I have a > > > server with 128G RAM and sufficient CPU's. Which configuration is better > > > a) two nodes on server each with ES\_HEAP\_SIZE say about 31G. b) one node > > > on the server with ES\_HEAP\_SIZE say 64G. > > > Thanks, > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/qsWdzyqclMc/unsubscribe](https://groups.google.com/d/topic/elasticsearch/qsWdzyqclMc/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CANBTPCFjoR9Y%3Dg\_Aqwd4RsbgTnefrp\_yNcf36kZp82nY6Am9JA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CANBTPCFjoR9Y%3Dg_Aqwd4RsbgTnefrp_yNcf36kZp82nY6Am9JA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [September 3, 2014, 6:47pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/6 "2014-09-03T18:47:24Z")
There are certain scalability issues with the garbage collection in the JVM so if you want to run large heaps with low latency you must expect extra challenges regarding stability and performance. For example, G1 GC is a garbage collector that works better than standard CMS GC at large heaps \> 20 GB. For heaps larger than 100 GB and \<10ms pause times, you can not expect much with the current Oracle JVM. There is Project Shenandoah for minimizing the stop-of-the-world stalls for such large heaps: [JEP 189: Shenandoah: A Low-Pause-Time Garbage Collector (Experimental)](http://openjdk.java.net/jeps/189) ES scales best over multiple machines horizontally, not vertically. More RAM does not automatically mean better performance at linear scale at a certain point - it depends on the JVM if it can keep up. I do not recommend running more than one JVM per machine, not because of fault tolerance, but just because of the overhead. The OS kernel must map the virtual memory (file system cache) and the thread scheduler to more than one VM and that steals quite few resources under heavy load. Jörg On Wed, Sep 3, 2014 at 7:43 PM, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) wrote: > Thanks for answering my questions. I also followed all these interesting > reading and learned a lot. > Cheers, > Jack > > Jinyuan (Jack) Zhou > > On Wed, Sep 3, 2014 at 10:02 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > > > The actual limitation in Java is compressed pointers: > > > > [Java ™ HotSpot Virtual Machine Performance Enhancements](http://docs.oracle.com/javase/7/docs/technotes/guides/vm/performance-enhancements-7.html#compressedOop) > > > > Thankfully Elasticsearch can run multiple nodes on the same server. Just > > pay attention to the direct size (off heap memory), mlockall settings and > > Lucene merge settings, as well as the allocation property referenced by > > David. Elasticsearch scales quite nicely vertically, so I would explore > > that option if possible. > > > > Cheers, > > > > Ivan > > > > On Wed, Sep 3, 2014 at 9:40 AM, vineeth mohan [vm.vineethmohan@gmail.com](mailto:vm.vineethmohan@gmail.com) > > wrote: > > > > > Hi , > > > > > > I am not sure if there is anything related to ES that makes a high RAM > > > problematic , but in general very high memory is not advised in java > > > systems. One of the reasons being the cobra effect which stems from the > > > fact that garbage collection might need more time for higher memory. > > > > > > COBRA EFFECT - > > > [https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect](https://plumbr.eu/blog/increasing-heap-size-beware-of-the-cobra-effect) > > > > > > Thanks > > > Vineeth > > > > > > On Wed, Sep 3, 2014 at 10:04 PM, Jinyuan Zhou [zhou.jinyuan@gmail.com](mailto:zhou.jinyuan@gmail.com) > > > wrote: > > > > > > > I read somewhere that the ES\_HEAP\_SIZE is best to be less than 31G, in > > > > this case JVM can use an 32 bit number to address memory locations. If > > > > my server have about 64G ram. This seems a perfect. But what If I have a > > > > server with 128G RAM and sufficient CPU's. Which configuration is better > > > > a) two nodes on server each with ES\_HEAP\_SIZE say about 31G. b) one node > > > > on the server with ES\_HEAP\_SIZE say 64G. > > > > Thanks, > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/03180561-2e7d-4e04-ab97-9d8b3a922f86%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAGdPd5noOXMuVBZLoz9dJjxko5z4LmWBdk6aSTY2W2sUW7MbuA%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/qsWdzyqclMc/unsubscribe](https://groups.google.com/d/topic/elasticsearch/qsWdzyqclMc/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQCF25cqzkhZ%2B2ooPfjFMyiSonWdKN%2BfPaiJvkEimK7Y0Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CANBTPCFjoR9Y%3Dg\_Aqwd4RsbgTnefrp\_yNcf36kZp82nY6Am9JA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CANBTPCFjoR9Y%3Dg_Aqwd4RsbgTnefrp_yNcf36kZp82nY6Am9JA%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CANBTPCFjoR9Y%3Dg\_Aqwd4RsbgTnefrp\_yNcf36kZp82nY6Am9JA%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CANBTPCFjoR9Y%3Dg_Aqwd4RsbgTnefrp_yNcf36kZp82nY6Am9JA%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHnCaeYJp4%2BJKkViFRKQjANTkt\_Q%3Dxb\_U7QihYRb3fjrQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHnCaeYJp4%2BJKkViFRKQjANTkt_Q%3Dxb_U7QihYRb3fjrQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [September 4, 2014, 4:24pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/7 "2014-09-04T16:24:13Z")
On Wed, Sep 3, 2014 at 11:47 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > ES scales best over multiple machines horizontally, not vertically. More > RAM does not automatically mean better performance at linear scale at a > certain point - it depends on the JVM if it can keep up. Ouch, what a brainfart I had! Yes, I meant to say horizontally as well. If you shard your indices correctly, or use time-based indices, you can always increase capacity at a later point. The shard allocator does not take into consideration the difference in computing power between machines, so it is best to have all nodes be relatively the same (unless you do manual allocation). Most clustered software such as Hadoop work in such as fashion. -- Ivan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDfLn\_Zc-VJ4fakTJNS4%2Bur9k9K4ymeJC8CWjZ1NrHbvg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDfLn_Zc-VJ4fakTJNS4%2Bur9k9K4ymeJC8CWjZ1NrHbvg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Jinyuan\_Zhou](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jinyuan_zhou/32/90185_2.png) [@Jinyuan\_Zhou](https://discuss.elastic.co/u/Jinyuan_Zhou)\ **Post date:** [September 4, 2014, 6:11pm UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/8 "2014-09-04T18:11:47Z")
Say if I have two boxes for data node in the cluster: A and B. However, A is doubles capacity of B. Now I can start two data nodes on A and 1 on B. With the help of rack id attribute, ES will make sure replicate and its primary are not located on the same box. In this case, It seems to me that Machine A will get more shards than Machine B. Jinyuan (Jack) Zhou On Thu, Sep 4, 2014 at 9:24 AM, Ivan Brusic [ivan@brusic.com](mailto:ivan@brusic.com) wrote: > On Wed, Sep 3, 2014 at 11:47 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \< > [joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote: > > > ES scales best over multiple machines horizontally, not vertically. More > > RAM does not automatically mean better performance at linear scale at a > > certain point - it depends on the JVM if it can keep up. > > Ouch, what a brainfart I had! Yes, I meant to say horizontally as well. If > you shard your indices correctly, or use time-based indices, you can always > increase capacity at a later point. The shard allocator does not take into > consideration the difference in computing power between machines, so it is > best to have all nodes be relatively the same (unless you do manual > allocation). Most clustered software such as Hadoop work in such as fashion. > > -- > Ivan > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/qsWdzyqclMc/unsubscribe](https://groups.google.com/d/topic/elasticsearch/qsWdzyqclMc/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDfLn\_Zc-VJ4fakTJNS4%2Bur9k9K4ymeJC8CWjZ1NrHbvg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDfLn_Zc-VJ4fakTJNS4%2Bur9k9K4ymeJC8CWjZ1NrHbvg%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDfLn\_Zc-VJ4fakTJNS4%2Bur9k9K4ymeJC8CWjZ1NrHbvg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDfLn_Zc-VJ4fakTJNS4%2Bur9k9K4ymeJC8CWjZ1NrHbvg%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CANBTPCG2Y0YrG5dsc7DZm%2BHVxRO4PxEa6jWaXUsrLRTFd1Rw4Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CANBTPCG2Y0YrG5dsc7DZm%2BHVxRO4PxEa6jWaXUsrLRTFd1Rw4Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:04am UTC](https://discuss.elastic.co/t/should-es-heap-size-be-less-than-31g/19595/9 "2017-07-06T01:04:19Z")