# ES 2.2.0 JVM overflow

**URL:** <https://discuss.elastic.co/t/es-2-2-0-jvm-overflow/47104>\
**Category:** Elasticsearch\
**Created:** [April 12, 2016, 10:25am UTC](https://discuss.elastic.co/t/es-2-2-0-jvm-overflow/47104 "2016-04-12T10:25:11Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![radap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radap/32/8037_2.png) [@radap](https://discuss.elastic.co/u/radap)\
**Post date:** [April 12, 2016, 10:25am UTC](https://discuss.elastic.co/t/es-2-2-0-jvm-overflow/47104/1 "2016-04-12T10:25:11Z")

</div>

I`m using elastic 2.2.0 with the following configuration:

> cluster.name: test\_cluster  
> node.name: test\_node\_1  
> path.data: /mnt/elasticsearch/data  
> path.logs: /var/log/elasticsearch

> network.bind\_host:
> 
> - _local_
> - _eth0:ipv4_

> network.publish\_host: _eth0:ipv4_  
> network.host: _eth0:ipv4_

> discovery.zen.ping.unicast.hosts: ["x:9301"]  
> discovery.zen.ping\_timeout: 30s  
> bootstrap.mlockall: true

Also I have the following configuration in /etc/default/elasticsearch:

> ES\_HEAP\_SIZE=4g  
> MAX\_LOCKED\_MEMORY=unlimited

When I using Kibana to query the cluster (with only 1 node) after some heavy queries, I rapidly get to the point where no more queries could be processed and the cluster is super slow. I get **"heap\_used\_percent": 99** and the GC time is very high:

```
> b]->[3.8gb]/[3.9gb], all_pools {[young] [101.1mb]->[12.5mb]/[133.1mb]}{[survivor] [16.6mb]->[0b]/[16.6mb]}{[old] [3.8gb]->[3.8gb]/[3.8gb]}
> [2016-04-12 13:08:05,445][INFO][monitor.jvm] [test_node_1] [gc][old][1115][72] duration [8.9s], collections [1]/[9.9s], total [8.9s]/[4.9m], memory [3.9gb]->[3.8gb]/[3.9gb], all_pools {[young] [121.1mb]->[2.8mb]/[133.1mb]}{[survivor] [16.6mb]->[0b]/[16.6mb]}{[old] [3.8gb]->[3.8gb]/[3.8gb]}
> [2016-04-12 13:08:18,375][INFO][monitor.jvm] [test_node_1] [gc][old][1119][73] duration [9.1s], collections [1]/[9.9s], total [9.1s]/[5m], memory [3.8gb]->[3.8gb]/[3.9gb], all_pools {[young] [58.4mb]->[1.3mb]/[133.1mb]}{[survivor] [14.6mb]->[0b]/[16.6mb]}{[old] [3.8gb]->[3.8gb]/[3.8gb]}

```

What is the problem? I set 4g heap size (the machine have 8g in total) and I can see **it is not a swappiness problem because the I/O is low**. Why is the cluster slows down so much after a couple of queries in kibana?

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 12, 2016, 10:42am UTC](https://discuss.elastic.co/t/es-2-2-0-jvm-overflow/47104/2 "2016-04-12T10:42:04Z")

</div>

What sort of queries are they?

---

<div class="post-metadata">

**Author:** ![radap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radap/32/8037_2.png) [@radap](https://discuss.elastic.co/u/radap)\
**Post date:** [April 12, 2016, 10:50am UTC](https://discuss.elastic.co/t/es-2-2-0-jvm-overflow/47104/3 "2016-04-12T10:50:19Z")

</div>

just regular visualizations like " triggeredBy: "DATA\_T" AND network.x: 425 AND connectivity.type:"MOBILE" " nothing spacial really. The problem is that the es (seems like) do not release the memory back and stuck processing something long after the query returns- the cpu just keep being high. I can see it in Grafana (using collectd on the es machine).

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [April 12, 2016, 10:08pm UTC](https://discuss.elastic.co/t/es-2-2-0-jvm-overflow/47104/4 "2016-04-12T22:08:02Z")

</div>

That looks ok, how much data do you have in the cluster?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 10:59pm UTC](https://discuss.elastic.co/t/es-2-2-0-jvm-overflow/47104/5 "2017-07-05T22:59:50Z")

</div>