# JVM heap usage issue due to percolate queries

**URL:** <https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610>\
**Category:** Elasticsearch\
**Created:** [April 7, 2017, 1:01pm UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610 "2017-04-07T13:01:47Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Anatoly\_Petkevich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anatoly_petkevich/32/97607_2.png) [@Anatoly\_Petkevich](https://discuss.elastic.co/u/Anatoly_Petkevich)\
**Post date:** [April 7, 2017, 1:01pm UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/1 "2017-04-07T13:01:47Z")

</div>

Hi,  
Document percolation in our environment periodically causes high JVM usage with a steady grow and reaches max memory limit.  
We cannot figure out what causes this issue, but it looks like as a memory leak related to dandling in-memory temporary Lucene indices used by percolator.  
Many input document have nested fields.

ES version is 5.2.2.  
Thank you.

 ![](https://us1.discourse-cdn.com/elastic/original/3X/6/f/6ff0f93b3d2078c836ca37764972b7bd94222740.JPG)

 ![](https://us1.discourse-cdn.com/elastic/original/3X/d/4/d4d6fe3d90ba344f3d6f9c8cad7c1dd90479ba6c.JPG)

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [April 7, 2017, 7:39pm UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/2 "2017-04-07T19:39:15Z")

</div>

Hello Anatoly,

In 5.x, the class `org.elasticsearch.index.cache.bitset.BitsetFilterCache` is actually the [node query cache](https://www.elastic.co/guide/en/elasticsearch/reference/5.3/query-cache.html). Indeed ~75% is too high (default is 10%).

Check if your elasticsearch.yml sets `indices.queries.cache.size`.

Regards

---

<div class="post-metadata">

**Author:** ![nimes](https://avatars.discourse-cdn.com/v4/letter/n/9f8e36/32.png) [@nimes](https://discuss.elastic.co/u/nimes)\
**Post date:** [April 8, 2017, 11:19am UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/3 "2017-04-08T11:19:14Z")

</div>

Thiago,

We don't set `indices.queries.cache.size`, leaving it by default.  
The issue affects only nodes having percolate indices (see heap dump from one of those nodes in Anatoly's post above).  
The heap dump below is from one of the healthy nodes in the same cluster that don't have percolate indices:

 ![](https://us1.discourse-cdn.com/elastic/original/3X/1/9/199a0e52530955cc9661bf02ce8eface941dc429.png)

Thank you

---

<div class="post-metadata">

**Author:** ![thiago](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/thiago/32/32096_2.png) [@thiago](https://discuss.elastic.co/u/thiago)\
**Post date:** [April 8, 2017, 11:36am UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/4 "2017-04-08T11:36:45Z")

</div>

Hello,

Please attach your complete elasticsearch.yml

Cheers

---

<div class="post-metadata">

**Author:** ![nimes](https://avatars.discourse-cdn.com/v4/letter/n/9f8e36/32.png) [@nimes](https://discuss.elastic.co/u/nimes)\
**Post date:** [April 8, 2017, 11:52am UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/5 "2017-04-08T11:52:47Z")

</div>

Thiago,

See below `elasticsearch.yml` from our test cluster where the issue is reproduced (some settings not related to the issue were omitted, like discovery, xpack etc.):

```
bootstrap.memory_lock: true
indices.memory.index_buffer_size: 30%
indices.memory.min_index_buffer_size: 96mb
thread_pool.bulk.size: '3'
thread_pool.bulk.queue_size: "-1"

```

Thanks

---

<div class="post-metadata">

**Author:** ![Anatoly\_Petkevich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anatoly_petkevich/32/97607_2.png) [@Anatoly\_Petkevich](https://discuss.elastic.co/u/Anatoly_Petkevich)\
**Post date:** [April 10, 2017, 5:20am UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/6 "2017-04-10T05:20:04Z")

</div>

Hello,  
Could be this issue related to [[Percolator] Remove caching and support "now" in range queries](https://github.com/elastic/elasticsearch/issues/23859) ?

---

<div class="post-metadata">

**Author:** ![Anatoly\_Petkevich](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anatoly_petkevich/32/97607_2.png) [@Anatoly\_Petkevich](https://discuss.elastic.co/u/Anatoly_Petkevich)\
**Post date:** [April 17, 2017, 5:54am UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/7 "2017-04-17T05:54:26Z")

</div>

A bug [https://github.com/elastic/elasticsearch/issues/24108](https://github.com/elastic/elasticsearch/issues/24108) has been created

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 15, 2017, 6:06am UTC](https://discuss.elastic.co/t/jvm-heap-usage-issue-due-to-percolate-queries/81610/8 "2017-05-15T06:06:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.