# Service heap size configuration **URL:** \ **Category:** Elasticsearch\ **Created:** [July 23, 2013, 11:05am UTC](https://discuss.elastic.co/t/service-heap-size-configuration/12896 "2013-07-23T11:05:56Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![dkoum](https://avatars.discourse-cdn.com/v4/letter/d/9dc877/32.png) [@dkoum](https://discuss.elastic.co/u/dkoum)\ **Post date:** [July 23, 2013, 11:05am UTC](https://discuss.elastic.co/t/service-heap-size-configuration/12896/1 "2013-07-23T11:05:56Z")
Hi all, I have set up a two-node elasticsearch cluster (0.20.6) under centos and have been managing it using the service wrapper. I need to increase the heap size to deal with an increase in index count. I have set the environment variables ES\_MIN\_MEM and ES\_MAX\_MEM to 1280 in /etc/profiles. The command set|grep ES\_ confirms that the values are set. I have also set ES\_HEAP\_SIZE=1280 in the service wrapper's own elasticsearch.conf. Using bigdesk, Ι can see that the allocated heap has risen as expected. Still, the memory used by es is kept under 1GB. That is, gc is still called under the 1GB limit. Am I missing something? Dionisis Koumouras -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [July 23, 2013, 12:54pm UTC](https://discuss.elastic.co/t/service-heap-size-configuration/12896/2 "2013-07-23T12:54:12Z")
I'm not familiar with bigdesk but I can tell you that java applications don't fill their entire heap before garbage collecting for the first time. They more stuff from pool to pool as they garbage collect. The first thing I'd check to make sure your changes took effect is the resident memory size of the application. I usually do `ps aux | grep elasticsearch` and check the sixth column. That should be around 120% of what you set ES\_HEAP\_SIZE to. When java applications run out of memory they spend forever doing full garbage collections. You can check the number of full garbage collections that your application has done as well as how much time it has spent on those collections like so `sudo jstat -gc `. The FGC and FGCT columns are what you are looking for if those are going up quickly your application is out of heap space. Nik On Tue, Jul 23, 2013 at 7:05 AM, Dionisis Koumouras [kumdio@gmail.com](mailto:kumdio@gmail.com)wrote: > Hi all, > I have set up a two-node elasticsearch cluster (0.20.6) under centos and > have been managing it using the service wrapper. I need to increase the > heap size to deal with an increase in index count. > > I have set the environment variables ES\_MIN\_MEM and ES\_MAX\_MEM to 1280 in > /etc/profiles. The command set|grep ES\_ confirms that the values are set. > > I have also set ES\_HEAP\_SIZE=1280 in the service wrapper's own > elasticsearch.conf. > > Using bigdesk, Ι can see that the allocated heap has risen as expected. > Still, the memory used by es is kept under 1GB. That is, gc is still called > under the 1GB limit. Am I missing something? > > Dionisis Koumouras > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dkoum](https://avatars.discourse-cdn.com/v4/letter/d/9dc877/32.png) [@dkoum](https://discuss.elastic.co/u/dkoum)\ **Post date:** [July 23, 2013, 2:18pm UTC](https://discuss.elastic.co/t/service-heap-size-configuration/12896/3 "2013-07-23T14:18:56Z")
thanks for the tips Nikolas. Following your advice, ps reported that the jvm consumes 1152648 kB, less than what it's supposed to, I guess. top also reports 1.1g as resident memory. I tried the nodes stats api. Here's a part of the output: "jvm" : { "timestamp" : 1374586439575, "uptime" : "3 hours, 11 minutes, 54 seconds and 188 milliseconds", "uptime\_in\_millis" : 11514188, "mem" : { "heap\_used" : "856mb", "heap\_used\_in\_bytes" : 897619760, "heap\_committed" : "1.2gb", "heap\_committed\_in\_bytes" : 1307312128, "non\_heap\_used" : "37.5mb", "non\_heap\_used\_in\_bytes" : 39423472, "non\_heap\_committed" : "58mb", "non\_heap\_committed\_in\_bytes" : 60837888, So, bigdesk is in par with the nodes stats api (I guess that's where it gets the data it presents). Still, I'm confused since they both report different amounts of memory from ps/top. If the ps reading is correct, it would justify the fact that all garbage collections start under the previous 1GB limit. It could, of course, also mean that es does not need the extra memory just yet. Garbage collections only start twice a minute and have lasted 14secs during 3 hours of operation. On Tue, Jul 23, 2013 at 3:54 PM, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > I'm not familiar with bigdesk but I can tell you that java applications > don't fill their entire heap before garbage collecting for the first time. > They more stuff from pool to pool as they garbage collect. > > The first thing I'd check to make sure your changes took effect is the > resident memory size of the application. I usually do `ps aux | grep elasticsearch` and check the sixth column. That should be around 120% of > what you set ES\_HEAP\_SIZE to. > > When java applications run out of memory they spend forever doing full > garbage collections. You can check the number of full garbage collections > that your application has done as well as how much time it has spent on > those collections like so `sudo jstat -gc `. The FGC and FGCT columns > are what you are looking for if those are going up quickly your application > is out of heap space. > > Nik > > On Tue, Jul 23, 2013 at 7:05 AM, Dionisis Koumouras [kumdio@gmail.com](mailto:kumdio@gmail.com)wrote: > > > Hi all, > > I have set up a two-node elasticsearch cluster (0.20.6) under centos and > > have been managing it using the service wrapper. I need to increase the > > heap size to deal with an increase in index count. > > > > I have set the environment variables ES\_MIN\_MEM and ES\_MAX\_MEM to 1280 in > > /etc/profiles. The command set|grep ES\_ confirms that the values are set. > > > > I have also set ES\_HEAP\_SIZE=1280 in the service wrapper's own > > elasticsearch.conf. > > > > Using bigdesk, Ι can see that the allocated heap has risen as expected. > > Still, the memory used by es is kept under 1GB. That is, gc is still called > > under the 1GB limit. Am I missing something? > > > > Dionisis Koumouras > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [July 23, 2013, 2:33pm UTC](https://discuss.elastic.co/t/service-heap-size-configuration/12896/4 "2013-07-23T14:33:03Z")
14secs in 3 hours means everything is a-ok! You may want to check `ps aux | grep elasticsearch` and see if you are missing a -Xms parameter that matches your -Xmx. I generally like it better when they match because the application allocates all the heap up front so there are less surprises. On Tue, Jul 23, 2013 at 10:18 AM, Dionisis Koumouras [kumdio@gmail.com](mailto:kumdio@gmail.com)wrote: > thanks for the tips Nikolas. > > Following your advice, ps reported that the jvm consumes 1152648 kB, less > than what it's supposed to, I guess. top also reports 1.1g as resident > memory. > > I tried the nodes stats api. Here's a part of the output: > > "jvm" : { > "timestamp" : 1374586439575, > "uptime" : "3 hours, 11 minutes, 54 seconds and 188 milliseconds", > "uptime\_in\_millis" : 11514188, > "mem" : { > "heap\_used" : "856mb", > "heap\_used\_in\_bytes" : 897619760, > "heap\_committed" : "1.2gb", > "heap\_committed\_in\_bytes" : 1307312128, > "non\_heap\_used" : "37.5mb", > "non\_heap\_used\_in\_bytes" : 39423472, > "non\_heap\_committed" : "58mb", > "non\_heap\_committed\_in\_bytes" : 60837888, > > So, bigdesk is in par with the nodes stats api (I guess that's where it > gets the data it presents). Still, I'm confused since they both report > different amounts of memory from ps/top. If the ps reading is correct, it > would justify the fact that all garbage collections start under the > previous 1GB limit. > > It could, of course, also mean that es does not need the extra memory just > yet. Garbage collections only start twice a minute and have lasted 14secs > during 3 hours of operation. > > On Tue, Jul 23, 2013 at 3:54 PM, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com)wrote: > > > I'm not familiar with bigdesk but I can tell you that java applications > > don't fill their entire heap before garbage collecting for the first time. > > They more stuff from pool to pool as they garbage collect. > > > > The first thing I'd check to make sure your changes took effect is the > > resident memory size of the application. I usually do `ps aux | grep elasticsearch` and check the sixth column. That should be around 120% of > > what you set ES\_HEAP\_SIZE to. > > > > When java applications run out of memory they spend forever doing full > > garbage collections. You can check the number of full garbage collections > > that your application has done as well as how much time it has spent on > > those collections like so `sudo jstat -gc `. The FGC and FGCT columns > > are what you are looking for if those are going up quickly your application > > is out of heap space. > > > > Nik > > > > On Tue, Jul 23, 2013 at 7:05 AM, Dionisis Koumouras [kumdio@gmail.com](mailto:kumdio@gmail.com)wrote: > > > > > Hi all, > > > I have set up a two-node elasticsearch cluster (0.20.6) under centos and > > > have been managing it using the service wrapper. I need to increase the > > > heap size to deal with an increase in index count. > > > > > > I have set the environment variables ES\_MIN\_MEM and ES\_MAX\_MEM to 1280 > > > in /etc/profiles. The command set|grep ES\_ confirms that the values are > > > set. > > > > > > I have also set ES\_HEAP\_SIZE=1280 in the service wrapper's own > > > elasticsearch.conf. > > > > > > Using bigdesk, Ι can see that the allocated heap has risen as expected. > > > Still, the memory used by es is kept under 1GB. That is, gc is still called > > > under the 1GB limit. Am I missing something? > > > > > > Dionisis Koumouras > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [July 23, 2013, 11:47pm UTC](https://discuss.elastic.co/t/service-heap-size-configuration/12896/5 "2013-07-23T23:47:22Z")
The Java heap is broken into different generations: Perm, Old and Young. GC can happen at different times for different generations. You can spend days learning about JVM heaps. As Nikolas said: you are a-ok. -- Ivan On Tue, Jul 23, 2013 at 7:33 AM, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > 14secs in 3 hours means everything is a-ok! You may want to check `ps aux | grep elasticsearch` and see if you are missing a -Xms parameter that > matches your -Xmx. I generally like it better when they match because the > application allocates all the heap up front so there are less surprises. > > On Tue, Jul 23, 2013 at 10:18 AM, Dionisis Koumouras [kumdio@gmail.com](mailto:kumdio@gmail.com)wrote: > > > thanks for the tips Nikolas. > > > > Following your advice, ps reported that the jvm consumes 1152648 kB, less > > than what it's supposed to, I guess. top also reports 1.1g as resident > > memory. > > > > I tried the nodes stats api. Here's a part of the output: > > > > "jvm" : { > > "timestamp" : 1374586439575, > > "uptime" : "3 hours, 11 minutes, 54 seconds and 188 milliseconds", > > "uptime\_in\_millis" : 11514188, > > "mem" : { > > "heap\_used" : "856mb", > > "heap\_used\_in\_bytes" : 897619760, > > "heap\_committed" : "1.2gb", > > "heap\_committed\_in\_bytes" : 1307312128, > > "non\_heap\_used" : "37.5mb", > > "non\_heap\_used\_in\_bytes" : 39423472, > > "non\_heap\_committed" : "58mb", > > "non\_heap\_committed\_in\_bytes" : 60837888, > > > > So, bigdesk is in par with the nodes stats api (I guess that's where it > > gets the data it presents). Still, I'm confused since they both report > > different amounts of memory from ps/top. If the ps reading is correct, it > > would justify the fact that all garbage collections start under the > > previous 1GB limit. > > > > It could, of course, also mean that es does not need the extra memory > > just yet. Garbage collections only start twice a minute and have lasted > > 14secs during 3 hours of operation. > > > > On Tue, Jul 23, 2013 at 3:54 PM, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com)wrote: > > > > > I'm not familiar with bigdesk but I can tell you that java applications > > > don't fill their entire heap before garbage collecting for the first time. > > > They more stuff from pool to pool as they garbage collect. > > > > > > The first thing I'd check to make sure your changes took effect is the > > > resident memory size of the application. I usually do `ps aux | grep elasticsearch` and check the sixth column. That should be around 120% of > > > what you set ES\_HEAP\_SIZE to. > > > > > > When java applications run out of memory they spend forever doing full > > > garbage collections. You can check the number of full garbage collections > > > that your application has done as well as how much time it has spent on > > > those collections like so `sudo jstat -gc `. The FGC and FGCT columns > > > are what you are looking for if those are going up quickly your application > > > is out of heap space. > > > > > > Nik > > > > > > On Tue, Jul 23, 2013 at 7:05 AM, Dionisis Koumouras [kumdio@gmail.com](mailto:kumdio@gmail.com)wrote: > > > > > > > Hi all, > > > > I have set up a two-node elasticsearch cluster (0.20.6) under centos > > > > and have been managing it using the service wrapper. I need to increase the > > > > heap size to deal with an increase in index count. > > > > > > > > I have set the environment variables ES\_MIN\_MEM and ES\_MAX\_MEM to 1280 > > > > in /etc/profiles. The command set|grep ES\_ confirms that the values are > > > > set. > > > > > > > > I have also set ES\_HEAP\_SIZE=1280 in the service wrapper's own > > > > elasticsearch.conf. > > > > > > > > Using bigdesk, Ι can see that the allocated heap has risen as expected. > > > > Still, the memory used by es is kept under 1GB. That is, gc is still called > > > > under the 1GB limit. Am I missing something? > > > > > > > > Dionisis Koumouras > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:24am UTC](https://discuss.elastic.co/t/service-heap-size-configuration/12896/6 "2017-07-06T02:24:54Z")