# Multiple nodes on same machine **URL:** \ **Category:** Elasticsearch\ **Created:** [March 19, 2013, 11:35pm UTC](https://discuss.elastic.co/t/multiple-nodes-on-same-machine/11213 "2013-03-19T23:35:35Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Mingfeng\_Yang](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@Mingfeng\_Yang](https://discuss.elastic.co/u/Mingfeng_Yang)\ **Post date:** [March 19, 2013, 11:35pm UTC](https://discuss.elastic.co/t/multiple-nodes-on-same-machine/11213/1 "2013-03-19T23:35:35Z")
I plan to use elasticsearch as documentation retrieval engine which will serve hundreds of millions of documents, but the query rate will be low. The ES cluster will probably receive a few queries only each hour. We are planning to use ec2 m2.2xlarge instance, each with 32G memory and 4 CPU cores, so I like to run 4 ES nodes on each ec2 instance to maximize the CPU utilization rate. In this case, is it beneficial to run multiple nodes on same machine? My own experience with Solr is that it does help to use resources more efficiently. Regards, Ming -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 20, 2013, 12:18am UTC](https://discuss.elastic.co/t/multiple-nodes-on-same-machine/11213/2 "2013-03-20T00:18:02Z")
No, it is not beneficial. Here are the reasons: a) if you start many JVMs, you create a JVM-induced overhead. That is, JVMs compete for the resources the OS provide (CPU, network, memory). Because the OS must decide which JVM does get which resources, it takes more time and space to make decisions, and this is not negelectible. The more JVMs you execute in parallel, the higher the risk of overall system degradation and in many cases the risk of paging (swapping) is higher. b) the ES code is optimized for scalability. What does that mean? You can increase the parameters for CPU (threads), memory (heap) and network (netty pools) for the ES JVM and this increases the overall power as much as your machine can get along with it. There is no reason why you should not dedicate a whole machine to one single ES node. c) a single ES JVM can manage hundreds or thousands of Lucene indexes at once. This is done by index sharding and automatic workload distribution. Each node can hold many indices with many index shards. An ES node does not restrict you to a model of a single index with a single shard. Jörg Am 20.03.13 00:35, schrieb [mfyang@wisewindow.com](mailto:mfyang@wisewindow.com): > In this case, is it beneficial to run multiple nodes on same machine? > > My own experience with Solr is that it does help to use resources more > efficiently. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Mingfeng\_Yang](https://avatars.discourse-cdn.com/v4/letter/m/e480ec/32.png) [@Mingfeng\_Yang](https://discuss.elastic.co/u/Mingfeng_Yang)\ **Post date:** [March 20, 2013, 1:08am UTC](https://discuss.elastic.co/t/multiple-nodes-on-same-machine/11213/3 "2013-03-20T01:08:56Z")
Jorg, Thanks for the info, very useful. So basically I can run one ES instance which holds multiple shards, and once each shard gets big, I can migrate them to separate machines? Thanks, Ming On Tuesday, March 19, 2013 5:18:02 PM UTC-7, Jörg Prante wrote: > No, it is not beneficial. > > Here are the reasons: > > a) if you start many JVMs, you create a JVM-induced overhead. That is, > JVMs compete for the resources the OS provide (CPU, network, memory). > Because the OS must decide which JVM does get which resources, it takes > more time and space to make decisions, and this is not negelectible. The > more JVMs you execute in parallel, the higher the risk of overall system > degradation and in many cases the risk of paging (swapping) is higher. > > b) the ES code is optimized for scalability. What does that mean? You > can increase the parameters for CPU (threads), memory (heap) and network > (netty pools) for the ES JVM and this increases the overall power as > much as your machine can get along with it. There is no reason why you > should not dedicate a whole machine to one single ES node. > > c) a single ES JVM can manage hundreds or thousands of Lucene indexes at > once. This is done by index sharding and automatic workload > distribution. Each node can hold many indices with many index shards. An > ES node does not restrict you to a model of a single index with a single > shard. > > Jörg > > Am 20.03.13 00:35, schrieb [mfy...@wisewindow.com](mailto:mfy...@wisewindow.com) \: > > > In this case, is it beneficial to run multiple nodes on same machine? > > > > My own experience with Solr is that it does help to use resources more > > efficiently. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Andy\_Wick](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andy_wick/32/44017_2.png) [@Andy\_Wick](https://discuss.elastic.co/u/Andy_Wick)\ **Post date:** [March 20, 2013, 12:43pm UTC](https://discuss.elastic.co/t/multiple-nodes-on-same-machine/11213/4 "2013-03-20T12:43:44Z")
Totally agree with 32G machines, but as memory gets cheaper and cheaper I'm curious if anyone has actually done any benchmarking or stress tests on the single vs multi node with large memory machines. We actually run 2 nodes (22G each) on our 64G machines. a) so we can have -XX:+UseCompressedOops b) with the theory (untested) that GC pauses will be faster/less often/... On Tuesday, March 19, 2013 8:18:02 PM UTC-4, Jörg Prante wrote: > No, it is not beneficial. > > Here are the reasons: > > a) if you start many JVMs, you create a JVM-induced overhead. That is, > JVMs compete for the resources the OS provide (CPU, network, memory). > Because the OS must decide which JVM does get which resources, it takes > more time and space to make decisions, and this is not negelectible. The > more JVMs you execute in parallel, the higher the risk of overall system > degradation and in many cases the risk of paging (swapping) is higher. > > b) the ES code is optimized for scalability. What does that mean? You > can increase the parameters for CPU (threads), memory (heap) and network > (netty pools) for the ES JVM and this increases the overall power as > much as your machine can get along with it. There is no reason why you > should not dedicate a whole machine to one single ES node. > > c) a single ES JVM can manage hundreds or thousands of Lucene indexes at > once. This is done by index sharding and automatic workload > distribution. Each node can hold many indices with many index shards. An > ES node does not restrict you to a model of a single index with a single > shard. > > Jörg > > Am 20.03.13 00:35, schrieb [mfy...@wisewindow.com](mailto:mfy...@wisewindow.com) \: > > > In this case, is it beneficial to run multiple nodes on same machine? > > > > My own experience with Solr is that it does help to use resources more > > efficiently. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![vineeth\_mohan](https://avatars.discourse-cdn.com/v4/letter/v/bc79bd/32.png) [@vineeth\_mohan](https://discuss.elastic.co/u/vineeth_mohan)\ **Post date:** [March 20, 2013, 1:19pm UTC](https://discuss.elastic.co/t/multiple-nodes-on-same-machine/11213/5 "2013-03-20T13:19:34Z")
I have heard that multi core is maximum utlized with different process rather than different threads. If that is true and if the machine has many cores , wont muliple instance be a good idea ? Thanks Vineeth On Wed, Mar 20, 2013 at 6:13 PM, Andy Wick [andywick@gmail.com](mailto:andywick@gmail.com) wrote: > Totally agree with 32G machines, but as memory gets cheaper and cheaper > I'm curious if anyone has actually done any benchmarking or stress tests on > the single vs multi node with large memory machines. > > We actually run 2 nodes (22G each) on our 64G machines. > > a) so we can have -XX:+UseCompressedOops > b) with the theory (untested) that GC pauses will be faster/less often/... > > On Tuesday, March 19, 2013 8:18:02 PM UTC-4, Jörg Prante wrote: > > > No, it is not beneficial. > > > > Here are the reasons: > > > > a) if you start many JVMs, you create a JVM-induced overhead. That is, > > JVMs compete for the resources the OS provide (CPU, network, memory). > > Because the OS must decide which JVM does get which resources, it takes > > more time and space to make decisions, and this is not negelectible. The > > more JVMs you execute in parallel, the higher the risk of overall system > > degradation and in many cases the risk of paging (swapping) is higher. > > > > b) the ES code is optimized for scalability. What does that mean? You > > can increase the parameters for CPU (threads), memory (heap) and network > > (netty pools) for the ES JVM and this increases the overall power as > > much as your machine can get along with it. There is no reason why you > > should not dedicate a whole machine to one single ES node. > > > > c) a single ES JVM can manage hundreds or thousands of Lucene indexes at > > once. This is done by index sharding and automatic workload > > distribution. Each node can hold many indices with many index shards. An > > ES node does not restrict you to a model of a single index with a single > > shard. > > > > Jörg > > > > Am 20.03.13 00:35, schrieb [mfy...@wisewindow.com](mailto:mfy...@wisewindow.com): > > > > > In this case, is it beneficial to run multiple nodes on same machine? > > > > > > My own experience with Solr is that it does help to use resources more > > > efficiently. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:45am UTC](https://discuss.elastic.co/t/multiple-nodes-on-same-machine/11213/6 "2017-07-06T02:45:36Z")