# JVM heap usage is high

**URL:** https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784
**Category:** Elasticsearch
**Created:** [November 28, 2019, 4:03am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784 "2019-11-28T04:03:21Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![chanakagl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chanakagl/32/54677_2.png) [@chanakagl](https://discuss.elastic.co/u/chanakagl)
#### Post date: [November 28, 2019, 4:03am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/1 "2019-11-28T04:03:21Z")

</div>

Hi,

We have 7 nodes cluster with 3 masters and 4 data nodes. We have one index with,  
4 shards,  
1 replica  
size - 3.5GB  
12894 documents  
which is getting 100/s search requests normally and indexing rate is very low. But when peak hours reached index is hitting around 500 - 600 /s search requests. When that happens JVM heap is going up and hitting 99% and then data node is going down.

Search query :-  
GET rxxxxxxxx\_pxxxxxxx/\_search  
{  
"size": 10000,  
"query": {  
"bool": {  
"must": [  
{  
"term": {  
"promotionDisplayStatus": {  
"value": true,  
"boost": 1  
}  
}  
},  
{  
"terms": {  
"conditionEntityFieldId": [6],  
"boost": 1  
}  
},  
{  
"range": {  
"promotionActiveUntil": {  
"from": "2019-11-20",  
"to": null,  
"include\_lower": true,  
"include\_upper": true,  
"boost": 1  
}  
}  
}  
],  
"adjust\_pure\_negative": true,  
"boost": 1  
}  
}  
}

Could you please advice me to where or which setting needs to be tweaked?

---

<div class="post-metadata">

### Author: ![Armin\_Braun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/armin_braun/32/20092_2.png) [@Armin\_Braun](https://discuss.elastic.co/u/Armin_Braun)
#### Post date: [November 28, 2019, 8:58am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/2 "2019-11-28T08:58:16Z")

</div>

Hi @chanakagl

to better help you, could you tell me which version of ES you're running?

---

<div class="post-metadata">

### Author: ![chanakagl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chanakagl/32/54677_2.png) [@chanakagl](https://discuss.elastic.co/u/chanakagl)
#### Post date: [November 28, 2019, 9:00am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/3 "2019-11-28T09:00:02Z")

</div>

version 6.8.2

---

<div class="post-metadata">

### Author: ![Armin\_Braun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/armin_braun/32/20092_2.png) [@Armin\_Braun](https://discuss.elastic.co/u/Armin_Braun)
#### Post date: [November 28, 2019, 9:23am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/4 "2019-11-28T09:23:46Z")

</div>

Thanks @chanakagl

You could try tweaking the circuit breaker settings, see documentation on them [here](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/circuit-breaker.html).

In particular a lower value for the in-flight circuit breaker or the parent circuit breaker might be helpful. You could also look into lowering the size of your search thread-pool, see docs on that [here] ([https://www.elastic.co/guide/en/elasticsearch/reference/6.8/modules-threadpool.html](https://www.elastic.co/guide/en/elasticsearch/reference/6.8/modules-threadpool.html)) from the defaults.

If that doesn't help increasing the heap size of your data node JVMs might be unavoidable to make `6.8.2` perform in this scenario. Another option though would be to upgrade to `7.x` which has strong circuit breaker logic in place through the [real-memory circuit breaker](https://www.elastic.co/guide/en/elasticsearch/reference/7.5/circuit-breaker.html).

---

<div class="post-metadata">

### Author: ![chanakagl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chanakagl/32/54677_2.png) [@chanakagl](https://discuss.elastic.co/u/chanakagl)
#### Post date: [November 28, 2019, 10:15am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/5 "2019-11-28T10:15:14Z")

</div>

Thanks for quick response. I'll have a look on those areas. Meantime if i reduce no of shard to 1 and increase replica to 2 or 3 since index size only around 3.5GB with less documents, would that help?

---

<div class="post-metadata">

### Author: ![Armin\_Braun](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/armin_braun/32/20092_2.png) [@Armin\_Braun](https://discuss.elastic.co/u/Armin_Braun)
#### Post date: [November 28, 2019, 10:19am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/6 "2019-11-28T10:19:34Z")

</div>

@chanakagl

If your shard size is indeed that small, yes moving to a single shard may be a good optimisation (especially when the indexing rate is low).  
You'd have to use `3` replicas though to make full use of your `4` data nodes if I understand your setup correctly (with `2` one of your nodes will just be unused).

---

<div class="post-metadata">

### Author: ![chanakagl](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chanakagl/32/54677_2.png) [@chanakagl](https://discuss.elastic.co/u/chanakagl)
#### Post date: [November 28, 2019, 10:20am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/7 "2019-11-28T10:20:55Z")

</div>

Thanks i'll make those changes and see. Thank you very much again..!

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 26, 2019, 10:20am UTC](https://discuss.elastic.co/t/jvm-heap-usage-is-high/209784/8 "2019-12-26T10:20:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
