# GC running early?

**URL:** https://discuss.elastic.co/t/gc-running-early/84059
**Category:** Elasticsearch
**Created:** [April 28, 2017, 8:50pm UTC](https://discuss.elastic.co/t/gc-running-early/84059 "2017-04-28T20:50:28Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![retzkek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/retzkek/32/9732_2.png) [@retzkek](https://discuss.elastic.co/u/retzkek)
#### Post date: [April 28, 2017, 8:50pm UTC](https://discuss.elastic.co/t/gc-running-early/84059/1 "2017-04-28T20:50:28Z")

</div>

I'm working on setting up a new cluster (new hardware), and I'm seeing unexpected heap behavior while doing some load testing. Elasticsearch 5.2.2, OpenJDK 8, SL7 (i.e. RHEL7). All nodes are configured with 16gb heap, verified in the logs and API, yet GC appears to be running around 1.5gb. This is under constant indexing, a mix of log-like and document-like (updating) behavior. One node stands out, but I don't see anything else unusual about it (it's not the master, and all nodes are configured identically with puppet). Any suggestions?

Thanks,  
Kevin

 ![](https://us1.discourse-cdn.com/elastic/original/3X/c/8/c862f3d5ace076cba98af84202860baa553d4927.png)

Complete JVM options:

```
-Dfile.encoding=UTF-8
-Dio.netty.noKeySetOptimization=true
-Dio.netty.noUnsafe=true
-Dio.netty.recycler.maxCapacityPerThread=0
-Djava.awt.headless=true
-Djava.awt.headless=true
-Djdk.io.permissionsUseCanonicalPath=true
-Djna.nosys=true
-Dlog4j.shutdownHookEnabled=false
-Dlog4j.skipJansi=true
-Dlog4j2.disable.jmx=true
-XX:+AlwaysPreTouch
-XX:+DisableExplicitGC
-XX:+HeapDumpOnOutOfMemoryError
-XX:+UseCMSInitiatingOccupancyOnly
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-Xms16g
-Xmx16g
-Xss1m
-server
```

---

<div class="post-metadata">

### Author: ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 30, 2017, 9:02am UTC](https://discuss.elastic.co/t/gc-running-early/84059/2 "2017-04-30T09:02:22Z")

</div>

Based on the graph it looks like your nodes are configured with 2GB heap, not 16GB. How did you install Elasticsearch? How are you starting it?

---

<div class="post-metadata">

### Author: ![retzkek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/retzkek/32/9732_2.png) [@retzkek](https://discuss.elastic.co/u/retzkek)
#### Post date: [May 1, 2017, 4:01pm UTC](https://discuss.elastic.co/t/gc-running-early/84059/3 "2017-05-01T16:01:24Z")

</div>

Right, hence my confusion. API clearly shows 16gb though [1]. The nodes were installed and configured with the elastic/elasticseach puppet forge module, and are run under systemd. Now that I've loaded more data the heap usage has gone up on the busier nodes [2], so maybe it was just a matter of getting more activity. This is a much larger cluster than we were using before.

[1]

```
name heap.current heap.max
esworker08 1.6gb 15.8gb
esworker26 1.5gb 15.8gb
esworker05 1.5gb 15.8gb
esworker31 2.4gb 15.8gb
esworker01 1.4gb 15.8gb
esworker19 1gb 15.8gb
esworker32 1.1gb 15.8gb
esworker13 2.2gb 15.8gb
esclient01 7gb 15.9gb
esworker15 1.1gb 15.8gb
esclient02 1gb 15.9gb
esworker24 2.2gb 15.8gb
esworker16 1.5gb 15.8gb
esworker06 1.1gb 15.8gb
esworker29 1.9gb 15.8gb
esworker09 1.9gb 15.8gb
esworker11 960mb 15.8gb
esworker20 856.4mb 15.8gb
esworker33 9.2gb 15.8gb
esworker28 1.2gb 15.8gb
esworker07 1.9gb 15.8gb
esworker03 1gb 15.8gb
esworker25 1.1gb 15.8gb
esworker14 832.5mb 15.8gb
esworker02 1.4gb 15.8gb

```

[2]

 ![](https://us1.discourse-cdn.com/elastic/original/3X/a/8/a8253bc9a8c9a8cbf9c4b4d71acd45689205e8ef.jpg)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 29, 2017, 4:02pm UTC](https://discuss.elastic.co/t/gc-running-early/84059/4 "2017-05-29T16:02:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
