# High memory usage on dedicated master nodes **URL:** \ **Category:** Elasticsearch\ **Created:** [July 16, 2014, 5:14pm UTC](https://discuss.elastic.co/t/high-memory-usage-on-dedicated-master-nodes/18706 "2014-07-16T17:14:34Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![David\_Smith\_2](https://avatars.discourse-cdn.com/v4/letter/d/a9a28c/32.png) [@David\_Smith\_2](https://discuss.elastic.co/u/David_Smith_2)\ **Post date:** [July 16, 2014, 5:14pm UTC](https://discuss.elastic.co/t/high-memory-usage-on-dedicated-master-nodes/18706/1 "2014-07-16T17:14:34Z")
We have cluster with 22 data nodes and 3 dedicated master nodes running ES 1.1.0. Data nodes have 16 GB of memory (half given to JVM heap) and the dedicated masters have 4 GB (half for heap). Data nodes have consistent memory usage (about 50-60%) but we're observing that the master node is constantly growing and shrinking it's memory use (after GC) every 5 minutes. See the bigdesk graph. [https://lh6.googleusercontent.com/-03s6N1l4oQ8/U8ayMdqO86I/AAAAAAAAABg/BWC91tetsvk/s1600/Screen+Shot+2014-07-16+at+1.04.25+PM.png](https://lh6.googleusercontent.com/-03s6N1l4oQ8/U8ayMdqO86I/AAAAAAAAABg/BWC91tetsvk/s1600/Screen+Shot+2014-07-16+at+1.04.25+PM.png) I'm curious as to why the master node is using up close to 1.5 GB of memory. I thought that maintaining cluster state shouldn't take up that much memory and master nodes should be able to work with much less memory. Anybody know why? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6a273add-a20b-41ab-9941-e6c9197389c0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6a273add-a20b-41ab-9941-e6c9197389c0%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 16, 2014, 11:43pm UTC](https://discuss.elastic.co/t/high-memory-usage-on-dedicated-master-nodes/18706/2 "2014-07-16T23:43:58Z")
Are they master only or are you sending queries through them as well? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 17 July 2014 03:14, David Smith [davidksmith2k@gmail.com](mailto:davidksmith2k@gmail.com) wrote: > We have cluster with 22 data nodes and 3 dedicated master nodes running ES > 1.1.0. Data nodes have 16 GB of memory (half given to JVM heap) and the > dedicated masters have 4 GB (half for heap). Data nodes have consistent > memory usage (about 50-60%) but we're observing that the master node is > constantly growing and shrinking it's memory use (after GC) every 5 > minutes. See the bigdesk graph. > > [https://lh6.googleusercontent.com/-03s6N1l4oQ8/U8ayMdqO86I/AAAAAAAAABg/BWC91tetsvk/s1600/Screen+Shot+2014-07-16+at+1.04.25+PM.png](https://lh6.googleusercontent.com/-03s6N1l4oQ8/U8ayMdqO86I/AAAAAAAAABg/BWC91tetsvk/s1600/Screen+Shot+2014-07-16+at+1.04.25+PM.png) > > I'm curious as to why the master node is using up close to 1.5 GB of > memory. I thought that maintaining cluster state shouldn't take up that > much memory and master nodes should be able to work with much less memory. > Anybody know why? > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6a273add-a20b-41ab-9941-e6c9197389c0%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6a273add-a20b-41ab-9941-e6c9197389c0%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/6a273add-a20b-41ab-9941-e6c9197389c0%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6a273add-a20b-41ab-9941-e6c9197389c0%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624Zem%3D3TnHCoi7Gta7rCsK%3DDiuwurjwHRxNQ6HLjho9Z1Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624Zem%3D3TnHCoi7Gta7rCsK%3DDiuwurjwHRxNQ6HLjho9Z1Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![smonasco\_2](https://avatars.discourse-cdn.com/v4/letter/s/848f3c/32.png) [@smonasco\_2](https://discuss.elastic.co/u/smonasco_2)\ **Post date:** [July 17, 2014, 4:07am UTC](https://discuss.elastic.co/t/high-memory-usage-on-dedicated-master-nodes/18706/3 "2014-07-17T04:07:45Z")
Maybe I'm missing something but if you give java mms em it will use it if only to store garbage. The trough after a garbage collection is usually more indicative of what is actually in use. This looks like a CMS/parnew setup. Parnew is fast and low blocking, but leaves stuff behind. CMS blocks but is really good at taking out the garbage. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0e7c621e-3e85-4bc3-ac0f-a1971cb097f6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0e7c621e-3e85-4bc3-ac0f-a1971cb097f6%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![David\_Smith\_2](https://avatars.discourse-cdn.com/v4/letter/d/a9a28c/32.png) [@David\_Smith\_2](https://discuss.elastic.co/u/David_Smith_2)\ **Post date:** [July 17, 2014, 2:35pm UTC](https://discuss.elastic.co/t/high-memory-usage-on-dedicated-master-nodes/18706/4 "2014-07-17T14:35:15Z")
They are dedicated masters and no queries are going through them. smonasco, that's it I believe. It's ParNew for young gen. I made a mistake in our puppet configs and gave the same amount of memory to both data nodes and master nodes for young generation (Xmn) even though master nodes only have 1/4 of the memory data nodes have. So all 2G of memory in these master nodes are in young generation. > On Jul 17, 2014, at 12:07 AM, smonasco [smonasco@gmail.com](mailto:smonasco@gmail.com) wrote: > > Maybe I'm missing something but if you give java mms em it will use it if only to store garbage. The trough after a garbage collection is usually more indicative of what is actually in use. > > This looks like a CMS/parnew setup. Parnew is fast and low blocking, but leaves stuff behind. CMS blocks but is really good at taking out the garbage. > > -- > You received this message because you are subscribed to the Google Groups "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0e7c621e-3e85-4bc3-ac0f-a1971cb097f6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0e7c621e-3e85-4bc3-ac0f-a1971cb097f6%40googlegroups.com). > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/A5405B23-A1AC-4ABC-B357-81B3977A5E39%40gmail.com](https://groups.google.com/d/msgid/elasticsearch/A5405B23-A1AC-4ABC-B357-81B3977A5E39%40gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:15am UTC](https://discuss.elastic.co/t/high-memory-usage-on-dedicated-master-nodes/18706/5 "2017-07-06T01:15:11Z")