# Idealized server setup? **URL:** \ **Category:** Elasticsearch\ **Created:** [December 9, 2013, 7:47pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775 "2013-12-09T19:47:44Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Josh\_Harrison](https://avatars.discourse-cdn.com/v4/letter/j/0ea827/32.png) [@Josh\_Harrison](https://discuss.elastic.co/u/Josh_Harrison)\ **Post date:** [December 9, 2013, 7:47pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/1 "2013-12-09T19:47:44Z")
When designing a dedicated ES cluster, does it make more sense to have two or three servers with a ton of resources each, or 5-10 cheaper commodity hardware systems? I know when an http request comes into a given machine, it can automatically be routed to another. Does this routing happen over 9200 or 9300? I'm picturing having the cheap machines connected to both our internal network for 9200, and to each other on a private 10.0.0.0 network for 9300. Would that result in a performance boost? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8bd2d826-549e-472b-b5bf-249a3e1c9767%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8bd2d826-549e-472b-b5bf-249a3e1c9767%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [December 9, 2013, 7:53pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/2 "2013-12-09T19:53:07Z")
On Mon, Dec 9, 2013 at 2:47 PM, Josh Harrison [hijakk@gmail.com](mailto:hijakk@gmail.com) wrote: > When designing a dedicated ES cluster, does it make more sense to have two > or three servers with a ton of resources each, or 5-10 cheaper commodity > hardware systems? I'd go commodity to get more ram. The sweet spot (I've heard) is 64gb per machine with each one running with a 30gb heap. > I know when an http request comes into a given machine, it can > automatically be routed to another. Does this routing happen over 9200 or > 9300? 9300 normally. If you run more than one ES on the node on will run on 9301, I believe. > I'm picturing having the cheap machines connected to both our internal > network for 9200, and to each other on a private 10.0.0.0 network for 9300. > Would that result in a performance boost? I think there are too many variables here. So long as you have fast switched connections between the machines you _should_ be ok. Elasticsearch currently doesn't know how to handle nodes of significantly different power automatically. It won't balance automatically based on machine power but there are things you can do to help that some. > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/8bd2d826-549e-472b-b5bf-249a3e1c9767%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8bd2d826-549e-472b-b5bf-249a3e1c9767%40googlegroups.com) > . > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd03KCW%3DjYQxbu1y\_c1Na36JNdJ%3DWf\_ipa1sR9gTssj%3D%3DA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd03KCW%3DjYQxbu1y_c1Na36JNdJ%3DWf_ipa1sR9gTssj%3D%3DA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [December 9, 2013, 9:09pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/3 "2013-12-09T21:09:39Z")
No more than 64G (with a 32G heap) is best as java doesn't compress pointers over 32g, which means you lose out. We were running a cluster of 8 nodes with 512G _per node_ and the resource wastage was immense, as was the GC! Not to mention java wouldn't even run with a 256G heap size. The docs state that ES will listen on any IP ( [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-network.html)), but that it will take the first one it finds. So you can't have it listening on multiple addresses. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 10 December 2013 06:53, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > On Mon, Dec 9, 2013 at 2:47 PM, Josh Harrison [hijakk@gmail.com](mailto:hijakk@gmail.com) wrote: > > > When designing a dedicated ES cluster, does it make more sense to have > > two or three servers with a ton of resources each, or 5-10 cheaper > > commodity hardware systems? > > I'd go commodity to get more ram. The sweet spot (I've heard) is 64gb per > machine with each one running with a 30gb heap. > > > I know when an http request comes into a given machine, it can > > automatically be routed to another. Does this routing happen over 9200 or > > 9300? > > 9300 normally. If you run more than one ES on the node on will run on > 9301, I believe. > > > I'm picturing having the cheap machines connected to both our internal > > network for 9200, and to each other on a private 10.0.0.0 network for 9300. > > Would that result in a performance boost? > > I think there are too many variables here. So long as you have fast > switched connections between the machines you _should_ be ok. > > Elasticsearch currently doesn't know how to handle nodes of significantly > different power automatically. It won't balance automatically based on > machine power but there are things you can do to help that some. > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/8bd2d826-549e-472b-b5bf-249a3e1c9767%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8bd2d826-549e-472b-b5bf-249a3e1c9767%40googlegroups.com) > > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd03KCW%3DjYQxbu1y\_c1Na36JNdJ%3DWf\_ipa1sR9gTssj%3D%3DA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd03KCW%3DjYQxbu1y_c1Na36JNdJ%3DWf_ipa1sR9gTssj%3D%3DA%40mail.gmail.com) > . > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624ZvhjtHHqrU4%3Dn5kxsS2dCqAPj1FMJchm6nd-W2Ccn9jw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624ZvhjtHHqrU4%3Dn5kxsS2dCqAPj1FMJchm6nd-W2Ccn9jw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [December 9, 2013, 11:47pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/4 "2013-12-09T23:47:28Z")
What performance do you ask for: maximum speed for executing a query? Or maximum throughput of the overall system for all queries? In general, ES is not designed for vertical scaling on few big oomph machines. ES design for scaling out horizontally over lots of commodity machines of same type. Note that you can not get faster the more machines you add, but you get higher overall throughput. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHWtw\_vb1GABZ5fsZDsdHO9m%2BrbYO3GUAFjVnzmmKHFvw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHWtw_vb1GABZ5fsZDsdHO9m%2BrbYO3GUAFjVnzmmKHFvw%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Josh\_Harrison](https://avatars.discourse-cdn.com/v4/letter/j/0ea827/32.png) [@Josh\_Harrison](https://discuss.elastic.co/u/Josh_Harrison)\ **Post date:** [December 10, 2013, 5:07pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/5 "2013-12-10T17:07:52Z")
Great, thanks all. Better throughput is the goal. I'll have to see if I can scrounge some decent systems up! On Monday, December 9, 2013 3:47:28 PM UTC-8, Jörg Prante wrote: > What performance do you ask for: maximum speed for executing a query? Or > maximum throughput of the overall system for all queries? > > In general, ES is not designed for vertical scaling on few big oomph > machines. ES design for scaling out horizontally over lots of commodity > machines of same type. Note that you can not get faster the more machines > you add, but you get higher overall throughput. > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e7d8df77-0874-47cb-896c-e0f4d06ab47e%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e7d8df77-0874-47cb-896c-e0f4d06ab47e%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Josh\_Harrison](https://avatars.discourse-cdn.com/v4/letter/j/0ea827/32.png) [@Josh\_Harrison](https://discuss.elastic.co/u/Josh_Harrison)\ **Post date:** [December 10, 2013, 6:50pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/6 "2013-12-10T18:50:11Z")
Hm, ok, so ES may not deal with substantially different capabilities of machines in terms of speed, but if I can throw a bunch of older systems with only a few GB of ram and a few hundred GB of storage space, is ES aware of the space constraints - distributing shards and replicas so that they don't hit the storage capacity limit right away? Thanks, Josh On Tuesday, December 10, 2013 9:07:52 AM UTC-8, Josh Harrison wrote: > Great, thanks all. Better throughput is the goal. I'll have to see if I > can scrounge some decent systems up! > > On Monday, December 9, 2013 3:47:28 PM UTC-8, Jörg Prante wrote: > > > What performance do you ask for: maximum speed for executing a query? Or > > maximum throughput of the overall system for all queries? > > > > In general, ES is not designed for vertical scaling on few big oomph > > machines. ES design for scaling out horizontally over lots of commodity > > machines of same type. Note that you can not get faster the more machines > > you add, but you get higher overall throughput. > > > > Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/31edc157-1487-4c39-b34c-d856b827f816%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/31edc157-1487-4c39-b34c-d856b827f816%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [December 10, 2013, 6:55pm UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/7 "2013-12-10T18:55:14Z")
On Tue, Dec 10, 2013 at 1:50 PM, Josh Harrison [hijakk@gmail.com](mailto:hijakk@gmail.com) wrote: > Hm, ok, so ES may not deal with substantially different capabilities of > machines in terms of speed, but if I can throw a bunch of older systems > with only a few GB of ram and a few hundred GB of storage space, is ES > aware of the space constraints - distributing shards and replicas so that > they don't hit the storage capacity limit right away? Hard disk, yes: > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Ram, I don't think so. Nik -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2N0Hnm8fjjL7bTaQzQgzwB\_B2Fcq2yos9kkTrAiYZ\_aA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd2N0Hnm8fjjL7bTaQzQgzwB_B2Fcq2yos9kkTrAiYZ_aA%40mail.gmail.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:02am UTC](https://discuss.elastic.co/t/idealized-server-setup/14775/8 "2017-07-06T02:02:08Z")