# Ideal setup for EC2 cluster Config **URL:** \ **Category:** Elasticsearch\ **Created:** [August 20, 2014, 5:30pm UTC](https://discuss.elastic.co/t/ideal-setup-for-ec2-cluster-config/19371 "2014-08-20T17:30:17Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![IronMike](https://avatars.discourse-cdn.com/v4/letter/i/e9a140/32.png) [@IronMike](https://discuss.elastic.co/u/IronMike)\ **Post date:** [August 20, 2014, 5:30pm UTC](https://discuss.elastic.co/t/ideal-setup-for-ec2-cluster-config/19371/1 "2014-08-20T17:30:17Z")
I know I have more reading to do, but some of the terms are still confusing to me. I would like to index about 5-6 million documents, doc is about 5k in size. Everything lives on the cloud, feeder instances and index instances. The 3 nodes are eligible to be master nodes. Should I make the three nodes masters, should they be "node.data" = true? Does making the 3 nodes masters reduce split brain problem? Can someone elaborate on what I need to be thinking about while trying to come up with the best configuration? Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/bfcfc42a-3e19-4bff-88b9-b6beb6044bde%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/bfcfc42a-3e19-4bff-88b9-b6beb6044bde%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![IronMike](https://avatars.discourse-cdn.com/v4/letter/i/e9a140/32.png) [@IronMike](https://discuss.elastic.co/u/IronMike)\ **Post date:** [August 20, 2014, 6:14pm UTC](https://discuss.elastic.co/t/ideal-setup-for-ec2-cluster-config/19371/2 "2014-08-20T18:14:10Z")
I also wanted to add, while reading up on this topic, dedicating a master only node kept popping up. With this configuration: node 1 {master=yes, node.data = false} node 2 {master=no, node.data = yes} node 3 {master=no, node.data = yes} What happens if node1 the master encounters a network issue? On Wednesday, August 20, 2014 1:30:17 PM UTC-4, IronMan2014 wrote: > I know I have more reading to do, but some of the terms are still > confusing to me. > I would like to index about 5-6 million documents, doc is about 5k in > size. > Everything lives on the cloud, feeder instances and index instances. > The 3 nodes are eligible to be master nodes. > > Should I make the three nodes masters, should they be "node.data" = true? > Does making the 3 nodes masters reduce split brain problem? > > Can someone elaborate on what I need to be thinking about while trying to > come up with the best configuration? > > Thanks -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [August 20, 2014, 9:11pm UTC](https://discuss.elastic.co/t/ideal-setup-for-ec2-cluster-config/19371/3 "2014-08-20T21:11:42Z")
If you only have 3 nodes, I would just stick to the defaults, which is both master and data. Having dedicated master (no data) nodes helps because it helps eliminate OOM pressures since the actual data lives elsewhere. With so few nodes, every machine should hold a portion of the data. Dedicated master nodes make sense when you have a bigger cluster IMHO. Besides the number of documents and their size, the most important metric is how many fields are indexed and what type. Dozens of analyzed string fields per document might encountered memory problems. Few fields and mainly numerical data? You would have no problems. I would start off with the defaults and only make changes if you notice some erroneous pattern. Cheers, Ivan On Wed, Aug 20, 2014 at 11:14 AM, IronMan2014 [sabdalla80@gmail.com](mailto:sabdalla80@gmail.com) wrote: > I also wanted to add, while reading up on this topic, dedicating a master > only node kept popping up. > > With this configuration: > node 1 {master=yes, node.data = false} > node 2 {master=no, node.data = yes} > node 3 {master=no, node.data = yes} > > What happens if node1 the master encounters a network issue? > > On Wednesday, August 20, 2014 1:30:17 PM UTC-4, IronMan2014 wrote: > > > I know I have more reading to do, but some of the terms are still > > confusing to me. > > I would like to index about 5-6 million documents, doc is about 5k in > > size. > > Everything lives on the cloud, feeder instances and index instances. > > The 3 nodes are eligible to be master nodes. > > > > Should I make the three nodes masters, should they be "node.data" = true? > > Does making the 3 nodes masters reduce split brain problem? > > > > Can someone elaborate on what I need to be thinking about while trying to > > come up with the best configuration? > > > > Thanks > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAYE1RUUWbJH%3Dw1UCpd9vQY\_3Tg9XDZ5t%3DhokmgLEfEyw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAYE1RUUWbJH%3Dw1UCpd9vQY_3Tg9XDZ5t%3DhokmgLEfEyw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ankit\_Jain\_2](https://avatars.discourse-cdn.com/v4/letter/a/d26b3c/32.png) [@Ankit\_Jain\_2](https://discuss.elastic.co/u/Ankit_Jain_2)\ **Post date:** [August 20, 2014, 9:29pm UTC](https://discuss.elastic.co/t/ideal-setup-for-ec2-cluster-config/19371/4 "2014-08-20T21:29:46Z")
So if you have a 9 node cluster... and you want to have 3 master eligible nodes... do you need 3 machines with master=yes, data=false? That would mean 2 machines just sit around doing nothing until something is wrong with the master. Is that correct? I'm trying to set up a large cluster with decent redundancy... Also, what's the benefit of 'query nodes': master=no, data=no? -Ankit On Wednesday, August 20, 2014 2:11:51 PM UTC-7, Ivan Brusic wrote: > If you only have 3 nodes, I would just stick to the defaults, which is > both master and data. > > Having dedicated master (no data) nodes helps because it helps eliminate > OOM pressures since the actual data lives elsewhere. With so few nodes, > every machine should hold a portion of the data. Dedicated master nodes > make sense when you have a bigger cluster IMHO. > > Besides the number of documents and their size, the most important metric > is how many fields are indexed and what type. Dozens of analyzed string > fields per document might encountered memory problems. Few fields and > mainly numerical data? You would have no problems. > > I would start off with the defaults and only make changes if you notice > some erroneous pattern. > > Cheers, > > Ivan > > On Wed, Aug 20, 2014 at 11:14 AM, IronMan2014 \<[sabda...@gmail.com](mailto:sabda...@gmail.com) > \\> wrote: > > > I also wanted to add, while reading up on this topic, dedicating a master > > only node kept popping up. > > > > With this configuration: > > node 1 {master=yes, node.data = false} > > node 2 {master=no, node.data = yes} > > node 3 {master=no, node.data = yes} > > > > What happens if node1 the master encounters a network issue? > > > > On Wednesday, August 20, 2014 1:30:17 PM UTC-4, IronMan2014 wrote: > > > > > I know I have more reading to do, but some of the terms are still > > > confusing to me. > > > I would like to index about 5-6 million documents, doc is about 5k in > > > size. > > > Everything lives on the cloud, feeder instances and index instances. > > > The 3 nodes are eligible to be master nodes. > > > > > > Should I make the three nodes masters, should they be "node.data" = > > > true? > > > Does making the 3 nodes masters reduce split brain problem? > > > > > > Can someone elaborate on what I need to be thinking about while trying > > > to come up with the best configuration? > > > > > > Thanks > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0a7071bd-e0c5-4436-875c-96f20a57a9ae%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0a7071bd-e0c5-4436-875c-96f20a57a9ae%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 20, 2014, 9:56pm UTC](https://discuss.elastic.co/t/ideal-setup-for-ec2-cluster-config/19371/5 "2014-08-20T21:56:13Z")
Yes, but there is nothing stopping you from sending queries to these master nodes if you wish. Client nodes can be useful if you do a lot of, or heavy queries. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 21 August 2014 07:29, Ankit Jain [ankit@quettra.com](mailto:ankit@quettra.com) wrote: > So if you have a 9 node cluster... and you want to have 3 master eligible > nodes... do you need 3 machines with master=yes, data=false? > > That would mean 2 machines just sit around doing nothing until something > is wrong with the master. Is that correct? I'm trying to set up a large > cluster with decent redundancy... > > Also, what's the benefit of 'query nodes': master=no, data=no? > > -Ankit > > On Wednesday, August 20, 2014 2:11:51 PM UTC-7, Ivan Brusic wrote: > > > If you only have 3 nodes, I would just stick to the defaults, which is > > both master and data. > > > > Having dedicated master (no data) nodes helps because it helps eliminate > > OOM pressures since the actual data lives elsewhere. With so few nodes, > > every machine should hold a portion of the data. Dedicated master nodes > > make sense when you have a bigger cluster IMHO. > > > > Besides the number of documents and their size, the most important metric > > is how many fields are indexed and what type. Dozens of analyzed string > > fields per document might encountered memory problems. Few fields and > > mainly numerical data? You would have no problems. > > > > I would start off with the defaults and only make changes if you notice > > some erroneous pattern. > > > > Cheers, > > > > Ivan > > > > On Wed, Aug 20, 2014 at 11:14 AM, IronMan2014 [sabda...@gmail.com](mailto:sabda...@gmail.com) wrote: > > > > > I also wanted to add, while reading up on this topic, dedicating a > > > master only node kept popping up. > > > > > > With this configuration: > > > node 1 {master=yes, node.data = false} > > > node 2 {master=no, node.data = yes} > > > node 3 {master=no, node.data = yes} > > > > > > What happens if node1 the master encounters a network issue? > > > > > > On Wednesday, August 20, 2014 1:30:17 PM UTC-4, IronMan2014 wrote: > > > > > > > I know I have more reading to do, but some of the terms are still > > > > confusing to me. > > > > I would like to index about 5-6 million documents, doc is about 5k in > > > > size. > > > > Everything lives on the cloud, feeder instances and index instances. > > > > The 3 nodes are eligible to be master nodes. > > > > > > > > Should I make the three nodes masters, should they be "node.data" = > > > > true? > > > > Does making the 3 nodes masters reduce split brain problem? > > > > > > > > Can someone elaborate on what I need to be thinking about while trying > > > > to come up with the best configuration? > > > > > > > > Thanks > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/177c46b5-29ce-4386-965b-619fa6ff7cb9%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/0a7071bd-e0c5-4436-875c-96f20a57a9ae%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0a7071bd-e0c5-4436-875c-96f20a57a9ae%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/0a7071bd-e0c5-4436-875c-96f20a57a9ae%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/0a7071bd-e0c5-4436-875c-96f20a57a9ae%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624Z6Aj5i8JrvKKQt0XUfJO2woi5cJnq4-e2%2B%2BF9gX7CvYw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Z6Aj5i8JrvKKQt0XUfJO2woi5cJnq4-e2%2B%2BF9gX7CvYw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:07am UTC](https://discuss.elastic.co/t/ideal-setup-for-ec2-cluster-config/19371/6 "2017-07-06T01:07:18Z")