# Master Node vs. Data Node Architecture

**URL:** <https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134>\
**Category:** Elasticsearch\
**Created:** [February 12, 2015, 2:08pm UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134 "2015-02-12T14:08:42Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric\_Luellen](https://avatars.discourse-cdn.com/v4/letter/e/ce73a5/32.png) [@Eric\_Luellen](https://discuss.elastic.co/u/Eric_Luellen)\
**Post date:** [February 12, 2015, 2:08pm UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/1 "2015-02-12T14:08:42Z")

</div>

Hello,

Currently I have a 3 node ElasticSearch cluster. Each node is a RHEL VM  
with 16 gig RAM. The basic config is:

- All nodes can be master and are data nodes.
- 3 shards and 1 replica
- 6 different indexes

I'm starting to run into issues of ElasticSearch bogging down on searches  
and is completely freezing sometimes at night. I've dedicated 9 gig to heap  
size and it says i'm using ~60% of the heap RAM and about 70% of the  
overall heap. So even though I'm using quite a bit of the heap, I'm not  
maxed out. I've attached a screenshot of the exact stats from Elastic HQ.  
I'm averaging around 10,000 events/sec coming into the cluster from 6  
different Logstash instances on another server.

My question is what can I do to help the stability and speed of my cluster.  
Currently I'm having issues with 1 node going down and it taking everything  
else down. The HA portion isn't working very well. I'm debating about  
either adding 1 more node with the exact same stats or adding 2 more  
smaller VMs that will act as master nodes only. I didn't know which one was  
recommended or where I would get the biggest bang for the buck.

Any information would be greatly appreciated.

Thanks,  
Eric

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\
**Post date:** [February 12, 2015, 2:15pm UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/2 "2015-02-12T14:15:15Z")

</div>

See this:

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

Basically, the recommended pattern talks about isolating responsibilities.  
A node should either be a data node, master-eligible node, or an external  
gateway to the cluster (client node)

--

Itamar Syn-Hershko  
[http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko)  
Freelance Developer & Consultant  
[Lucene.NET](http://Lucene.NET) committer and PMC member

On Thu, Feb 12, 2015 at 4:08 PM, Eric [eric.luellen@gmail.com](mailto:eric.luellen@gmail.com) wrote:

> Hello,
> 
> Currently I have a 3 node Elasticsearch cluster. Each node is a RHEL VM  
> with 16 gig RAM. The basic config is:
> 
> - All nodes can be master and are data nodes.
> - 3 shards and 1 replica
> - 6 different indexes
> 
> I'm starting to run into issues of Elasticsearch bogging down on searches  
> and is completely freezing sometimes at night. I've dedicated 9 gig to heap  
> size and it says i'm using ~60% of the heap RAM and about 70% of the  
> overall heap. So even though I'm using quite a bit of the heap, I'm not  
> maxed out. I've attached a screenshot of the exact stats from Elastic HQ.  
> I'm averaging around 10,000 events/sec coming into the cluster from 6  
> different Logstash instances on another server.
> 
> My question is what can I do to help the stability and speed of my  
> cluster. Currently I'm having issues with 1 node going down and it taking  
> everything else down. The HA portion isn't working very well. I'm debating  
> about either adding 1 more node with the exact same stats or adding 2 more  
> smaller VMs that will act as master nodes only. I didn't know which one was  
> recommended or where I would get the biggest bang for the buck.
> 
> Any information would be greatly appreciated.
> 
> Thanks,  
> Eric
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u\_V\_\_6w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 12, 2015, 9:40pm UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/3 "2015-02-12T21:40:33Z")

</div>

Except that is overkill when you only have 3 nodes.

How much data do you have in the cluster?

On 13 February 2015 at 01:15, Itamar Syn-Hershko [itamar@code972.com](mailto:itamar@code972.com) wrote:

> See this:  
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html)
> 
> Basically, the recommended pattern talks about isolating responsibilities.  
> A node should either be a data node, master-eligible node, or an external  
> gateway to the cluster (client node)
> 
> --
> 
> Itamar Syn-Hershko  
> [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko)  
> Freelance Developer & Consultant  
> [Lucene.NET](http://Lucene.NET) committer and PMC member
> 
> On Thu, Feb 12, 2015 at 4:08 PM, Eric [eric.luellen@gmail.com](mailto:eric.luellen@gmail.com) wrote:
> 
> > Hello,
> > 
> > Currently I have a 3 node Elasticsearch cluster. Each node is a RHEL VM  
> > with 16 gig RAM. The basic config is:
> > 
> > - All nodes can be master and are data nodes.
> > - 3 shards and 1 replica
> > - 6 different indexes
> > 
> > I'm starting to run into issues of Elasticsearch bogging down on searches  
> > and is completely freezing sometimes at night. I've dedicated 9 gig to heap  
> > size and it says i'm using ~60% of the heap RAM and about 70% of the  
> > overall heap. So even though I'm using quite a bit of the heap, I'm not  
> > maxed out. I've attached a screenshot of the exact stats from Elastic HQ.  
> > I'm averaging around 10,000 events/sec coming into the cluster from 6  
> > different Logstash instances on another server.
> > 
> > My question is what can I do to help the stability and speed of my  
> > cluster. Currently I'm having issues with 1 node going down and it taking  
> > everything else down. The HA portion isn't working very well. I'm debating  
> > about either adding 1 more node with the exact same stats or adding 2 more  
> > smaller VMs that will act as master nodes only. I didn't know which one was  
> > recommended or where I would get the biggest bang for the buck.
> > 
> > Any information would be greatly appreciated.
> > 
> > Thanks,  
> > Eric
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > .  
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u\_V\_\_6w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com)  
> [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u\_V\_\_6w%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X93EwqeGf9S4UpMvtJy3%2BmaAjovfVicj7LRHz%2BPyAbSug%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X93EwqeGf9S4UpMvtJy3%2BmaAjovfVicj7LRHz%2BPyAbSug%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Itamar\_Syn\_Hershko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/itamar_syn_hershko/32/725_2.png) [@Itamar\_Syn\_Hershko](https://discuss.elastic.co/u/Itamar_Syn_Hershko)\
**Post date:** [February 12, 2015, 11:32pm UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/4 "2015-02-12T23:32:15Z")

</div>

Depending why the node goes down - going mid-way with dedicated master  
nodes is sometimes the solution

And if this is due to massive use of aggregations, doc-values may be the  
answer (or larger heap, but that's costlier)

--

Itamar Syn-Hershko  
[http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko)  
Freelance Developer & Consultant  
[Lucene.NET](http://Lucene.NET) committer and PMC member

On Thu, Feb 12, 2015 at 11:40 PM, Mark Walkom [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote:

> Except that is overkill when you only have 3 nodes.
> 
> How much data do you have in the cluster?
> 
> On 13 February 2015 at 01:15, Itamar Syn-Hershko [itamar@code972.com](mailto:itamar@code972.com)  
> wrote:
> 
> > See this:  
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html)
> > 
> > Basically, the recommended pattern talks about isolating  
> > responsibilities. A node should either be a data node, master-eligible  
> > node, or an external gateway to the cluster (client node)
> > 
> > --
> > 
> > Itamar Syn-Hershko  
> > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko)  
> > Freelance Developer & Consultant  
> > [Lucene.NET](http://Lucene.NET) committer and PMC member
> > 
> > On Thu, Feb 12, 2015 at 4:08 PM, Eric [eric.luellen@gmail.com](mailto:eric.luellen@gmail.com) wrote:
> > 
> > > Hello,
> > > 
> > > Currently I have a 3 node Elasticsearch cluster. Each node is a RHEL VM  
> > > with 16 gig RAM. The basic config is:
> > > 
> > > - All nodes can be master and are data nodes.
> > > - 3 shards and 1 replica
> > > - 6 different indexes
> > > 
> > > I'm starting to run into issues of Elasticsearch bogging down on  
> > > searches and is completely freezing sometimes at night. I've dedicated 9  
> > > gig to heap size and it says i'm using ~60% of the heap RAM and about 70%  
> > > of the overall heap. So even though I'm using quite a bit of the heap, I'm  
> > > not maxed out. I've attached a screenshot of the exact stats from Elastic  
> > > HQ. I'm averaging around 10,000 events/sec coming into the cluster from 6  
> > > different Logstash instances on another server.
> > > 
> > > My question is what can I do to help the stability and speed of my  
> > > cluster. Currently I'm having issues with 1 node going down and it taking  
> > > everything else down. The HA portion isn't working very well. I'm debating  
> > > about either adding 1 more node with the exact same stats or adding 2 more  
> > > smaller VMs that will act as master nodes only. I didn't know which one was  
> > > recommended or where I would get the biggest bang for the buck.
> > > 
> > > Any information would be greatly appreciated.
> > > 
> > > Thanks,  
> > > Eric
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .  
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u\_V\_\_6w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u\_V\_\_6w%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> > .
> > 
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X93EwqeGf9S4UpMvtJy3%2BmaAjovfVicj7LRHz%2BPyAbSug%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X93EwqeGf9S4UpMvtJy3%2BmaAjovfVicj7LRHz%2BPyAbSug%40mail.gmail.com)  
> [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X93EwqeGf9S4UpMvtJy3%2BmaAjovfVicj7LRHz%2BPyAbSug%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X93EwqeGf9S4UpMvtJy3%2BmaAjovfVicj7LRHz%2BPyAbSug%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> .
> 
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZvFX-bEqpSEnh3EmdbzAwAhDLE7PYDivd5Q2VnFu\_xviA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZvFX-bEqpSEnh3EmdbzAwAhDLE7PYDivd5Q2VnFu_xviA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![sunner](https://avatars.discourse-cdn.com/v4/letter/s/e19b73/32.png) [@sunner](https://discuss.elastic.co/u/sunner)\
**Post date:** [February 13, 2015, 8:52am UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/5 "2015-02-13T08:52:40Z")

</div>

I might as well ask a related question...

We're in a similar position to the OP, expanding and rearchitecting a  
cluster that's currently undersized. We'll have our new cluster spread  
across two datacenters, so I'm trying to figure out how to setup the nodes.

We'll have 4 data nodes, the database will probably land somewhere around  
750-1000 GB, it's a Graylog2 cluster so I'm not sure how many shards we'll  
have in the end, but we'll go with 2 replicas per shard so that we never  
end up with one shard being in just one DC. Likewise the Graylog2 servers  
will be spread between the two DC's, two per DC.

How many masters would be recommended for this? My initial thought would be  
that two would be good enough. The two failure scenarios I see as most  
likely(or perhaps least unlikely if you're an optimist) are:  
Master ES crashes - Other master takes over.  
The link between the DC's goes down or flaps - Won't matter if we have 1 or  
2 masters per DC anyway.

So would 2 masters, one per DC be good enough for this setup?

Regards  
Johan

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/e8f9926e-2265-4682-84f7-f1e063781c16%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e8f9926e-2265-4682-84f7-f1e063781c16%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Eric\_Luellen](https://avatars.discourse-cdn.com/v4/letter/e/ce73a5/32.png) [@Eric\_Luellen](https://discuss.elastic.co/u/Eric_Luellen)\
**Post date:** [February 13, 2015, 1:19pm UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/6 "2015-02-13T13:19:54Z")

</div>

According to Elastic HQ I currently am keeping 12 days of logs

- 8 nodes (3 data nodes that can all be master and 5 Logstash)
- 399 shards
- 59 indices
- 1,540,343,998 documents
- 380 GB

On Thursday, February 12, 2015 at 4:41:01 PM UTC-5, Mark Walkom wrote:

> Except that is overkill when you only have 3 nodes.
> 
> How much data do you have in the cluster?
> 
> On 13 February 2015 at 01:15, Itamar Syn-Hershko \<[ita...@code972.com](mailto:ita...@code972.com)  
> \<javascript:\>\> wrote:
> 
> > See this:  
> > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-node.html)
> > 
> > Basically, the recommended pattern talks about isolating  
> > responsibilities. A node should either be a data node, master-eligible  
> > node, or an external gateway to the cluster (client node)
> > 
> > --
> > 
> > Itamar Syn-Hershko  
> > [http://code972.com](http://code972.com) | @synhershko [https://twitter.com/synhershko](https://twitter.com/synhershko)  
> > Freelance Developer & Consultant  
> > [Lucene.NET](http://Lucene.NET) committer and PMC member
> > 
> > On Thu, Feb 12, 2015 at 4:08 PM, Eric \<[eric.l...@gmail.com](mailto:eric.l...@gmail.com) \<javascript:\>\>  
> > wrote:
> > 
> > > Hello,
> > > 
> > > Currently I have a 3 node Elasticsearch cluster. Each node is a RHEL VM  
> > > with 16 gig RAM. The basic config is:
> > > 
> > > - All nodes can be master and are data nodes.
> > > - 3 shards and 1 replica
> > > - 6 different indexes
> > > 
> > > I'm starting to run into issues of Elasticsearch bogging down on  
> > > searches and is completely freezing sometimes at night. I've dedicated 9  
> > > gig to heap size and it says i'm using ~60% of the heap RAM and about 70%  
> > > of the overall heap. So even though I'm using quite a bit of the heap, I'm  
> > > not maxed out. I've attached a screenshot of the exact stats from Elastic  
> > > HQ. I'm averaging around 10,000 events/sec coming into the cluster from 6  
> > > different Logstash instances on another server.
> > > 
> > > My question is what can I do to help the stability and speed of my  
> > > cluster. Currently I'm having issues with 1 node going down and it taking  
> > > everything else down. The HA portion isn't working very well. I'm debating  
> > > about either adding 1 more node with the exact same stats or adding 2 more  
> > > smaller VMs that will act as master nodes only. I didn't know which one was  
> > > recommended or where I would get the biggest bang for the buck.
> > > 
> > > Any information would be greatly appreciated.
> > > 
> > > Thanks,  
> > > Eric
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google  
> > > Groups "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send  
> > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > > To view this discussion on the web visit  
> > > [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com)  
> > > [https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/354a2326-5532-4239-87ea-f02af64fe71f%40googlegroups.com?utm_medium=email&utm_source=footer)  
> > > .  
> > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \<javascript:\>.  
> > To view this discussion on the web visit  
> > [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u\_V\_\_6w%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com)  
> > [https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u\_V\_\_6w%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAHTr4ZureLROJMaO7gVprFjA2OmRZA0ZYyH1v%2Bges06u_V__6w%40mail.gmail.com?utm_medium=email&utm_source=footer)  
> > .
> > 
> > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/10133d13-ff4d-4798-80e2-4fffabfeb53c%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/10133d13-ff4d-4798-80e2-4fffabfeb53c%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [February 16, 2015, 8:03am UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/7 "2015-02-16T08:03:29Z")

</div>

You should really start your own thread for this (please) 🙂

On 13 February 2015 at 19:52, [sunner@sunner.com](mailto:sunner@sunner.com) wrote:

> I might as well ask a related question...
> 
> We're in a similar position to the OP, expanding and rearchitecting a  
> cluster that's currently undersized. We'll have our new cluster spread  
> across two datacenters, so I'm trying to figure out how to setup the nodes.
> 
> We'll have 4 data nodes, the database will probably land somewhere around  
> 750-1000 GB, it's a Graylog2 cluster so I'm not sure how many shards we'll  
> have in the end, but we'll go with 2 replicas per shard so that we never  
> end up with one shard being in just one DC. Likewise the Graylog2 servers  
> will be spread between the two DC's, two per DC.
> 
> How many masters would be recommended for this? My initial thought would  
> be that two would be good enough. The two failure scenarios I see as most  
> likely(or perhaps least unlikely if you're an optimist) are:  
> Master ES crashes - Other master takes over.  
> The link between the DC's goes down or flaps - Won't matter if we have 1  
> or 2 masters per DC anyway.
> 
> So would 2 masters, one per DC be good enough for this setup?
> 
> Regards  
> Johan
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> To view this discussion on the web visit  
> [https://groups.google.com/d/msgid/elasticsearch/e8f9926e-2265-4682-84f7-f1e063781c16%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/e8f9926e-2265-4682-84f7-f1e063781c16%40googlegroups.com)  
> [https://groups.google.com/d/msgid/elasticsearch/e8f9926e-2265-4682-84f7-f1e063781c16%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/e8f9926e-2265-4682-84f7-f1e063781c16%40googlegroups.com?utm_medium=email&utm_source=footer)  
> .  
> For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8e3X0P%2BuQHa280v0GW3ecKwTtJHH9XuCp4X2QNHwTTEA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8e3X0P%2BuQHa280v0GW3ecKwTtJHH9XuCp4X2QNHwTTEA%40mail.gmail.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 12:32am UTC](https://discuss.elastic.co/t/master-node-vs-data-node-architecture/22134/8 "2017-07-06T00:32:34Z")

</div>


