# Elastic search High threads , 100% utilization of non heap memory **URL:** \ **Category:** Elasticsearch\ **Created:** [May 23, 2014, 5:18am UTC](https://discuss.elastic.co/t/elastic-search-high-threads-100-utilization-of-non-heap-memory/17686 "2014-05-23T05:18:37Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![srikanth\_ramineni](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@srikanth\_ramineni](https://discuss.elastic.co/u/srikanth_ramineni)\ **Post date:** [May 23, 2014, 5:18am UTC](https://discuss.elastic.co/t/elastic-search-high-threads-100-utilization-of-non-heap-memory/17686/1 "2014-05-23T05:18:37Z")
Hi Team, We are experiencing issue with high usage of non heap memory and high thread count. Mostly we are seeing GC process is running. We are watching threads using big desk from past two days. Threads are reaching peak. We are not sure why it is reaching this much high. Two days back we ran optimizer manually. We restarted the servers, still threads are showing peak. Can you please suggest what might be the issues. Is there any way to debug this kind of issue. Most of the times we are getting node not reachable and timeout error. [7.9gb], all\_pools {[young] [3.4mb]-\>[64.4mb]/[266.2mb]}{[survivor] [33.2mb]-\>[33.2mb]/[33.2mb]}{[old] [2.1gb]-\>[2.2gb]/[7.6gb]} [2014-05-22 20:42:53,966][WARN][monitor.jvm] [Echo] [gc][young][2864][37] duration [2s], collections [1]/[2.7s], total [2s]/[23.1s], memory [2.4gb]-\>[2.4gb]/[7.9gb], all\_pools {[young] [104.9mb]-\>[3.6mb]/[266.2mb]}{[survivor] [33.2mb]-\>[33.2mb]/[33.2mb]}{[old] [2.2gb]-\>[2.4gb]/[7.6gb]} [2014-05-22 20:43:09,921][INFO][monitor.jvm] [Echo] [gc][young][2865][46] duration [6.6s], collections [9]/[15.9s], total [6.6s]/[29.7s], memory [2.4gb]-\>[2.6gb]/[7.9gb], all\_pools {[young] [3.6mb]-\>[98mb]/[266.2mb]}{[survivor] [33.2mb]-\>[23.9kb]/[33.2mb]}{[old] [2.4gb]-\>[2.5gb]/[7.6gb]} Below is our configuration. Having 4 node cluster each node having 16GB RAM -\> Elastic search allocated to 10GB RAM on each node. Total size of data indexed : 450 GB Each node having 350GB space Thanks, Srikanth. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [June 1, 2014, 8:13pm UTC](https://discuss.elastic.co/t/elastic-search-high-threads-100-utilization-of-non-heap-memory/17686/2 "2014-06-01T20:13:59Z")
Hey, you should use the nodes stats and nodes info APIs to monitor elasticsearch (or marvel, if its commercially viable for you). Also, where are you seing the 100% utilization? Not memory but CPU wise - might make sense to add some info here. --Alex On Fri, May 23, 2014 at 7:18 AM, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) wrote: > Hi Team, > > We are experiencing issue with high usage of non heap memory and high > thread count. Mostly we are seeing GC process is running. We are watching > threads using big desk from past two days. Threads are reaching peak. We > are not sure why it is reaching this much high. Two days back we ran > optimizer manually. We restarted the servers, still threads are showing > peak. Can you please suggest what might be the issues. Is there any way > to debug this kind of issue. Most of the times we are getting node not > reachable and timeout error. > > [7.9gb], all\_pools {[young] [3.4mb]-\>[64.4mb]/[266.2mb]}{[survivor] > [33.2mb]-\>[33.2mb]/[33.2mb]}{[old] [2.1gb]-\>[2.2gb]/[7.6gb]} > [2014-05-22 20:42:53,966][WARN][monitor.jvm] [Echo] > [gc][young][2864][37] duration [2s], collections [1]/[2.7s], total > [2s]/[23.1s], memory [2.4gb]-\>[2.4gb]/[7.9gb], all\_pools {[young] > [104.9mb]-\>[3.6mb]/[266.2mb]}{[survivor] [33.2mb]-\>[33.2mb]/[33.2mb]}{[old] > [2.2gb]-\>[2.4gb]/[7.6gb]} > [2014-05-22 20:43:09,921][INFO][monitor.jvm] [Echo] > [gc][young][2865][46] duration [6.6s], collections [9]/[15.9s], total > [6.6s]/[29.7s], memory [2.4gb]-\>[2.6gb]/[7.9gb], all\_pools {[young] > [3.6mb]-\>[98mb]/[266.2mb]}{[survivor] [33.2mb]-\>[23.9kb]/[33.2mb]}{[old] > [2.4gb]-\>[2.5gb]/[7.6gb]} > > Below is our configuration. > > Having 4 node cluster > each node having 16GB RAM -\> Elastic search allocated to 10GB RAM on each > node. > Total size of data indexed : 450 GB > Each node having 350GB space > > Thanks, > Srikanth. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-0Y1ZRYbTGXYfvGh9ygeG-Q0VyoUU%3DpwQ019SoY%2BgbOg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-0Y1ZRYbTGXYfvGh9ygeG-Q0VyoUU%3DpwQ019SoY%2BgbOg%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![srikanth\_ramineni](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@srikanth\_ramineni](https://discuss.elastic.co/u/srikanth_ramineni)\ **Post date:** [June 1, 2014, 11:09pm UTC](https://discuss.elastic.co/t/elastic-search-high-threads-100-utilization-of-non-heap-memory/17686/3 "2014-06-01T23:09:39Z")
Hi Alex, Thanks for your response. We are using big desk to monitor the stats.Please find enclosed documents having screen shots for one of my node. Hope enclosed screens shots help you to provide some inputs to us. Thanks, Srikanth. On Sun, Jun 1, 2014 at 1:13 PM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de) wrote: > Hey, > > you should use the nodes stats and nodes info APIs to monitor > elasticsearch (or marvel, if its commercially viable for you). Also, where > are you seing the 100% utilization? Not memory but CPU wise - might make > sense to add some info here. > > --Alex > > On Fri, May 23, 2014 at 7:18 AM, srikanth ramineni [ri.srikanth@gmail.com](mailto:ri.srikanth@gmail.com) > wrote: > > > Hi Team, > > > > We are experiencing issue with high usage of non heap memory and high > > thread count. Mostly we are seeing GC process is running. We are watching > > threads using big desk from past two days. Threads are reaching peak. We > > are not sure why it is reaching this much high. Two days back we ran > > optimizer manually. We restarted the servers, still threads are showing > > peak. Can you please suggest what might be the issues. Is there any way > > to debug this kind of issue. Most of the times we are getting node not > > reachable and timeout error. > > > > [7.9gb], all\_pools {[young] [3.4mb]-\>[64.4mb]/[266.2mb]}{[survivor] > > [33.2mb]-\>[33.2mb]/[33.2mb]}{[old] [2.1gb]-\>[2.2gb]/[7.6gb]} > > [2014-05-22 20:42:53,966][WARN][monitor.jvm] [Echo] > > [gc][young][2864][37] duration [2s], collections [1]/[2.7s], total > > [2s]/[23.1s], memory [2.4gb]-\>[2.4gb]/[7.9gb], all\_pools {[young] > > [104.9mb]-\>[3.6mb]/[266.2mb]}{[survivor] [33.2mb]-\>[33.2mb]/[33.2mb]}{[old] > > [2.2gb]-\>[2.4gb]/[7.6gb]} > > [2014-05-22 20:43:09,921][INFO][monitor.jvm] [Echo] > > [gc][young][2865][46] duration [6.6s], collections [9]/[15.9s], total > > [6.6s]/[29.7s], memory [2.4gb]-\>[2.6gb]/[7.9gb], all\_pools {[young] > > [3.6mb]-\>[98mb]/[266.2mb]}{[survivor] [33.2mb]-\>[23.9kb]/[33.2mb]}{[old] > > [2.4gb]-\>[2.5gb]/[7.6gb]} > > > > Below is our configuration. > > > > Having 4 node cluster > > each node having 16GB RAM -\> Elastic search allocated to 10GB RAM on > > each node. > > Total size of data indexed : 450 GB > > Each node having 350GB space > > > > Thanks, > > Srikanth. > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/af014404-0b15-4387-93ca-ec5d3bdc805f%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/EZydbbYvmXI/unsubscribe](https://groups.google.com/d/topic/elasticsearch/EZydbbYvmXI/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-0Y1ZRYbTGXYfvGh9ygeG-Q0VyoUU%3DpwQ019SoY%2BgbOg%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-0Y1ZRYbTGXYfvGh9ygeG-Q0VyoUU%3DpwQ019SoY%2BgbOg%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-0Y1ZRYbTGXYfvGh9ygeG-Q0VyoUU%3DpwQ019SoY%2BgbOg%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-0Y1ZRYbTGXYfvGh9ygeG-Q0VyoUU%3DpwQ019SoY%2BgbOg%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAGuNUAWDAPm0by0Be7H9wHHFmeoZAsfzZ72CJZyEPYdgBExrvA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAGuNUAWDAPm0by0Be7H9wHHFmeoZAsfzZ72CJZyEPYdgBExrvA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:25am UTC](https://discuss.elastic.co/t/elastic-search-high-threads-100-utilization-of-non-heap-memory/17686/4 "2017-07-06T01:25:21Z")