# Used heap behaves wired, JVM Heapsize problems

**URL:** <https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871>\
**Category:** Elasticsearch\
**Created:** [July 20, 2020, 9:45am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871 "2020-07-20T09:45:33Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [July 20, 2020, 9:45am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/1 "2020-07-20T09:45:33Z")

</div>

Hi,  
my used JVM Heap behaves very weird. I didn't change any settings. 2 Nodes work fine, the third has problems.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/4/e46c0c4add22c7ddf5df9f8bb70880566d296ce2.png)  
(Orange Line)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/9/2948ee727ce8978b08c7a2a328c1eb7c885ac9e0.png)  
Settings:

```auto
-Xms3g
-Xmx3g

```

The PC has a lot of free RAM left.  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/5/7534126a65b79358ef384711a0c1a0e86c68c478.png)

What is happening here?  
Thanks,  
_defalt_

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 20, 2020, 11:58pm UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/3 "2020-07-20T23:58:16Z")

</div>

It's not clear what problems you have here sorry.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 21, 2020, 5:08am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/4 "2020-07-21T05:08:57Z")

</div>

What is the output of the cluster stats API?

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [July 21, 2020, 6:12am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/5 "2020-07-21T06:12:56Z")

</div>

```auto
{
"_nodes": {
"total": 3,
"successful": 3,
"failed": 0
},
"cluster_name": "dc-world",
"cluster_uuid": "wLeROlF5Tx6OMYftWH6U3Q",
"timestamp": 1595311712215,
"status": "green",
"indices": {
"count": 45,
"shards": {
"total": 146,
"primaries": 73,
"replication": 1,
"index": {
"shards": {
"min": 2,
"max": 10,
"avg": 3.2444444444444445
},
"primaries": {
"min": 1,
"max": 5,
"avg": 1.6222222222222222
},
"replication": {
"min": 1,
"max": 1,
"avg": 1
}
}
},
"docs": {
"count": 818506112,
"deleted": 2226751
},
"store": {
"size_in_bytes": 497094462921
},
"fielddata": {
"memory_size_in_bytes": 48104,
"evictions": 0
},
"query_cache": {
"memory_size_in_bytes": 2539645,
"total_count": 908225,
"hit_count": 298795,
"miss_count": 609430,
"cache_size": 166,
"cache_count": 4786,
"evictions": 4620
},
"completion": {
"size_in_bytes": 0
},
"segments": {
"count": 1882,
"memory_in_bytes": 243464880,
"terms_memory_in_bytes": 4960864,
"stored_fields_memory_in_bytes": 231750528,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 267520,
"points_memory_in_bytes": 0,
"doc_values_memory_in_bytes": 6485968,
"index_writer_memory_in_bytes": 12315080,
"version_map_memory_in_bytes": 260,
"fixed_bit_set_memory_in_bytes": 1089496,
"max_unsafe_auto_id_timestamp": 1595289610086,
"file_sizes": {}
},
"mappings": {
"field_types": [
{
"name": "alias",
"count": 69,
"index_count": 1
},
{
"name": "binary",
"count": 13,
"index_count": 4
},
{
"name": "boolean",
"count": 195,
"index_count": 27
},
{
"name": "date",
"count": 247,
"index_count": 38
},
{
"name": "date_range",
"count": 1,
"index_count": 1
},
{
"name": "double",
"count": 1,
"index_count": 1
},
{
"name": "flattened",
"count": 3,
"index_count": 3
},
{
"name": "float",
"count": 205,
"index_count": 20
},
{
"name": "geo_point",
"count": 14,
"index_count": 2
},
{
"name": "geo_shape",
"count": 3,
"index_count": 3
},
{
"name": "half_float",
"count": 59,
"index_count": 15
},
{
"name": "integer",
"count": 235,
"index_count": 16
},
{
"name": "ip",
"count": 40,
"index_count": 2
},
{
"name": "keyword",
"count": 2581,
"index_count": 43
},
{
"name": "long",
"count": 1982,
"index_count": 32
},
{
"name": "nested",
"count": 45,
"index_count": 13
},
{
"name": "object",
"count": 1902,
"index_count": 31
},
{
"name": "text",
"count": 517,
"index_count": 34
}
]
},
"analysis": {
"char_filter_types": [
{
"name": "pattern_replace",
"count": 4,
"index_count": 2
}
],
"tokenizer_types": [],
"filter_types": [
{
"name": "pattern_capture",
"count": 1,
"index_count": 1
},
{
"name": "shingle",
"count": 2,
"index_count": 2
}
],
"analyzer_types": [
{
"name": "custom",
"count": 1,
"index_count": 1
}
],
"built_in_char_filters": [],
"built_in_tokenizers": [
{
"name": "uax_url_email",
"count": 1,
"index_count": 1
}
],
"built_in_filters": [
{
"name": "lowercase",
"count": 1,
"index_count": 1
},
{
"name": "unique",
"count": 1,
"index_count": 1
}
],
"built_in_analyzers": [
{
"name": "simple",
"count": 4,
"index_count": 1
}
]
}
},
"nodes": {
"count": {
"total": 3,
"coordinating_only": 0,
"data": 3,
"ingest": 3,
"master": 3,
"ml": 3,
"remote_cluster_client": 3,
"transform": 3,
"voting_only": 0
},
"versions": [
"7.8.0"
],
"os": {
"available_processors": 24,
"allocated_processors": 24,
"names": [
{
"name": "Windows 10",
"count": 2
},
{
"name": "Windows 7",
"count": 1
}
],
"pretty_names": [
{
"pretty_name": "Windows 7",
"count": 1
},
{
"pretty_name": "Windows 10",
"count": 2
}
],
"mem": {
"total_in_bytes": 42440687616,
"free_in_bytes": 19766992896,
"used_in_bytes": 22673694720,
"free_percent": 47,
"used_percent": 53
}
},
"process": {
"cpu": {
"percent": 1
},
"open_file_descriptors": {
"min": -1,
"max": -1,
"avg": 0
}
},
"jvm": {
"max_uptime_in_millis": 71124807,
"versions": [
{
"version": "14.0.1",
"vm_name": "OpenJDK 64-Bit Server VM",
"vm_version": "14.0.1+7",
"vm_vendor": "AdoptOpenJDK",
"bundled_jdk": true,
"using_bundled_jdk": true,
"count": 2
},
{
"version": "11.0.7",
"vm_name": "OpenJDK 64-Bit Server VM",
"vm_version": "11.0.7+10",
"vm_vendor": "AdoptOpenJDK",
"bundled_jdk": true,
"using_bundled_jdk": false,
"count": 1
}
],
"mem": {
"heap_used_in_bytes": 5258006344,
"heap_max_in_bytes": 9593946112
},
"threads": 317
},
"fs": {
"total_in_bytes": 2988491104256,
"free_in_bytes": 1531093913600,
"available_in_bytes": 1531093913600
},
"plugins": [],
"network_types": {
"transport_types": {
"security4": 3
},
"http_types": {
"security4": 3
}
},
"discovery_types": {
"zen": 3
},
"packaging_types": [
{
"flavor": "default",
"type": "zip",
"count": 3
}
],
"ingest": {
"number_of_pipelines": 2,
"processor_stats": {
"gsub": {
"count": 0,
"failed": 0,
"current": 0,
"time_in_millis": 0
},
"script": {
"count": 0,
"failed": 0,
"current": 0,
"time_in_millis": 0
}
}
}
}
}

```

Behavior still the same but it slowly increased from 500-1000MB to 1000-1500MB:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/d/0de64b027b4b938b4b1a81b027d6094472415fc8.png)

Thanks

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 21, 2020, 6:18am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/6 "2020-07-21T06:18:21Z")

</div>

You have one node running Java11 while the other run Java14. Can you check if the one behaving different is the Java11 one? Are there any differences in JVM options between the nodes?

I would recommend having all nodes run on the same Java version.

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [July 21, 2020, 7:20am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/8 "2020-07-21T07:20:11Z")

</div>

I spoke too soon! The faulty node works now but the other two now ran into the same problem:  
This was the faulty one:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/e/eeeffb4d9f4938952c57ef43ae7b59f9d0ae3a29.png)  
This one was fine before:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/0/c03b405b4379a180d63c44bfaeed9974ea5548f8.png)  
Third node:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/e/9eb19fcd10619ea5a00dc4b5d87f03205b32a50a.png)

Cluster stats:

```auto
{
"_nodes": {
"total": 3,
"successful": 3,
"failed": 0
},
"cluster_name": "dc-world",
"cluster_uuid": "wLeROlF5Tx6OMYftWH6U3Q",
"timestamp": 1595315980994,
"status": "green",
"indices": {
"count": 45,
"shards": {
"total": 146,
"primaries": 73,
"replication": 1,
"index": {
"shards": {
"min": 2,
"max": 10,
"avg": 3.2444444444444445
},
"primaries": {
"min": 1,
"max": 5,
"avg": 1.6222222222222222
},
"replication": {
"min": 1,
"max": 1,
"avg": 1
}
}
},
"docs": {
"count": 818531250,
"deleted": 2286159
},
"store": {
"size_in_bytes": 497196443225
},
"fielddata": {
"memory_size_in_bytes": 45144,
"evictions": 0
},
"query_cache": {
"memory_size_in_bytes": 25784721,
"total_count": 537148,
"hit_count": 150278,
"miss_count": 386870,
"cache_size": 1851,
"cache_count": 4741,
"evictions": 2890
},
"completion": {
"size_in_bytes": 0
},
"segments": {
"count": 1881,
"memory_in_bytes": 243418500,
"terms_memory_in_bytes": 4959392,
"stored_fields_memory_in_bytes": 231750008,
"term_vectors_memory_in_bytes": 0,
"norms_memory_in_bytes": 267520,
"points_memory_in_bytes": 0,
"doc_values_memory_in_bytes": 6441580,
"index_writer_memory_in_bytes": 5050760,
"version_map_memory_in_bytes": 0,
"fixed_bit_set_memory_in_bytes": 1068520,
"max_unsafe_auto_id_timestamp": 1595312772667,
"file_sizes": {}
},
"mappings": {
"field_types": [
{
"name": "alias",
"count": 69,
"index_count": 1
},
{
"name": "binary",
"count": 13,
"index_count": 4
},
{
"name": "boolean",
"count": 195,
"index_count": 27
},
{
"name": "date",
"count": 247,
"index_count": 38
},
{
"name": "date_range",
"count": 1,
"index_count": 1
},
{
"name": "double",
"count": 1,
"index_count": 1
},
{
"name": "flattened",
"count": 3,
"index_count": 3
},
{
"name": "float",
"count": 205,
"index_count": 20
},
{
"name": "geo_point",
"count": 14,
"index_count": 2
},
{
"name": "geo_shape",
"count": 3,
"index_count": 3
},
{
"name": "half_float",
"count": 59,
"index_count": 15
},
{
"name": "integer",
"count": 235,
"index_count": 16
},
{
"name": "ip",
"count": 40,
"index_count": 2
},
{
"name": "keyword",
"count": 2581,
"index_count": 43
},
{
"name": "long",
"count": 1982,
"index_count": 32
},
{
"name": "nested",
"count": 45,
"index_count": 13
},
{
"name": "object",
"count": 1902,
"index_count": 31
},
{
"name": "text",
"count": 517,
"index_count": 34
}
]
},
"analysis": {
"char_filter_types": [
{
"name": "pattern_replace",
"count": 4,
"index_count": 2
}
],
"tokenizer_types": [],
"filter_types": [
{
"name": "pattern_capture",
"count": 1,
"index_count": 1
},
{
"name": "shingle",
"count": 2,
"index_count": 2
}
],
"analyzer_types": [
{
"name": "custom",
"count": 1,
"index_count": 1
}
],
"built_in_char_filters": [],
"built_in_tokenizers": [
{
"name": "uax_url_email",
"count": 1,
"index_count": 1
}
],
"built_in_filters": [
{
"name": "lowercase",
"count": 1,
"index_count": 1
},
{
"name": "unique",
"count": 1,
"index_count": 1
}
],
"built_in_analyzers": [
{
"name": "simple",
"count": 4,
"index_count": 1
}
]
}
},
"nodes": {
"count": {
"total": 3,
"coordinating_only": 0,
"data": 3,
"ingest": 3,
"master": 3,
"ml": 3,
"remote_cluster_client": 3,
"transform": 3,
"voting_only": 0
},
"versions": [
"7.8.0"
],
"os": {
"available_processors": 24,
"allocated_processors": 24,
"names": [
{
"name": "Windows 10",
"count": 2
},
{
"name": "Windows 7",
"count": 1
}
],
"pretty_names": [
{
"pretty_name": "Windows 7",
"count": 1
},
{
"pretty_name": "Windows 10",
"count": 2
}
],
"mem": {
"total_in_bytes": 42440687616,
"free_in_bytes": 19452805120,
"used_in_bytes": 22987882496,
"free_percent": 46,
"used_percent": 54
}
},
"process": {
"cpu": {
"percent": 3
},
"open_file_descriptors": {
"min": -1,
"max": -1,
"avg": 0
}
},
"jvm": {
"max_uptime_in_millis": 75393104,
"versions": [
{
"version": "14.0.1",
"vm_name": "OpenJDK 64-Bit Server VM",
"vm_version": "14.0.1+7",
"vm_vendor": "AdoptOpenJDK",
"bundled_jdk": true,
"using_bundled_jdk": true,
"count": 3
}
],
"mem": {
"heap_used_in_bytes": 4168996024,
"heap_max_in_bytes": 9663676416
},
"threads": 311
},
"fs": {
"total_in_bytes": 2988491104256,
"free_in_bytes": 1531115847680,
"available_in_bytes": 1531115847680
},
"plugins": [],
"network_types": {
"transport_types": {
"security4": 3
},
"http_types": {
"security4": 3
}
},
"discovery_types": {
"zen": 3
},
"packaging_types": [
{
"flavor": "default",
"type": "zip",
"count": 3
}
],
"ingest": {
"number_of_pipelines": 2,
"processor_stats": {
"gsub": {
"count": 0,
"failed": 0,
"current": 0,
"time_in_millis": 0
},
"script": {
"count": 0,
"failed": 0,
"current": 0,
"time_in_millis": 0
}
}
}
}
}

```

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [July 21, 2020, 7:23am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/9 "2020-07-21T07:23:04Z")

</div>

What does your JVM.options files look like? Are you using the settings that come with Elasticsearch 7.8 on Java14?

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [July 21, 2020, 7:29am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/10 "2020-07-21T07:29:56Z")

</div>

JVM Node 1:

> **JVM**
>
> ```auto
> ## JVM configuration
> 
> ################################################################
> ## IMPORTANT: JVM heap size
> ################################################################
> ##
> ## You should always set the min and max JVM heap
> ## size to the same value. For example, to set
> ## the heap to 4 GB, set:
> ##
> ## -Xms4g
> ## -Xmx4g
> ##
> ## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
> ## for more information
> ##
> ################################################################
> 
> # Xms represents the initial size of total heap space
> # Xmx represents the maximum size of total heap space
> 
> -Xms3g
> -Xmx3g
> 
> ################################################################
> ## Expert settings
> ################################################################
> ##
> ## All settings below this section are considered
> ## expert settings. Don't tamper with them unless
> ## you understand what you are doing
> ##
> ################################################################
> 
> ## GC configuration
> 8-13:-XX:+UseConcMarkSweepGC
> 8-13:-XX:CMSInitiatingOccupancyFraction=75
> 8-13:-XX:+UseCMSInitiatingOccupancyOnly
> 
> ## G1GC Configuration
> # NOTE: G1 GC is only supported on JDK version 10 or later
> # to use G1GC, uncomment the next two lines and update the version on the
> # following three lines to your version of the JDK
> # 10-13:-XX:-UseConcMarkSweepGC
> # 10-13:-XX:-UseCMSInitiatingOccupancyOnly
> 14-:-XX:+UseG1GC
> 14-:-XX:G1ReservePercent=25
> 14-:-XX:InitiatingHeapOccupancyPercent=30
> 
> ## JVM temporary directory
> -Djava.io.tmpdir=${ES_TMPDIR}
> 
> ## heap dumps
> 
> # generate a heap dump when an allocation from the Java heap fails
> # heap dumps are created in the working directory of the JVM
> -XX:+HeapDumpOnOutOfMemoryError
> 
> # specify an alternative path for heap dumps; ensure the directory exists and
> # has sufficient space
> -XX:HeapDumpPath=data
> 
> # specify an alternative path for JVM fatal error logs
> -XX:ErrorFile=logs/hs_err_pid%p.log
> 
> ## JDK 8 GC logging
> 8:-XX:+PrintGCDetails
> 8:-XX:+PrintGCDateStamps
> 8:-XX:+PrintTenuringDistribution
> 8:-XX:+PrintGCApplicationStoppedTime
> 8:-Xloggc:logs/gc.log
> 8:-XX:+UseGCLogFileRotation
> 8:-XX:NumberOfGCLogFiles=32
> 8:-XX:GCLogFileSize=64m
> 
> # JDK 9+ GC logging
> 9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
> 
> ```

JVM on the node which had the problem (Haven't changed anything to fix the issue):

> **JVM**
>
> ```auto
> ## JVM configuration
> 
> ################################################################
> ## IMPORTANT: JVM heap size
> ################################################################
> ##
> ## You should always set the min and max JVM heap
> ## size to the same value. For example, to set
> ## the heap to 4 GB, set:
> ##
> ## -Xms4g
> ## -Xmx4g
> ##
> ## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
> ## for more information
> ##
> ################################################################
> 
> # Xms represents the initial size of total heap space
> # Xmx represents the maximum size of total heap space
> 
> -Xms3g
> -Xmx3g
> 
> ################################################################
> ## Expert settings
> ################################################################
> ##
> ## All settings below this section are considered
> ## expert settings. Don't tamper with them unless
> ## you understand what you are doing
> ##
> ################################################################
> 
> ## GC configuration
> 8-13:-XX:+UseConcMarkSweepGC
> 8-13:-XX:CMSInitiatingOccupancyFraction=75
> 8-13:-XX:+UseCMSInitiatingOccupancyOnly
> 
> ## G1GC Configuration
> # NOTE: G1 GC is only supported on JDK version 10 or later
> # to use G1GC, uncomment the next two lines and update the version on the
> # following three lines to your version of the JDK
> # 10-13:-XX:-UseConcMarkSweepGC
> # 10-13:-XX:-UseCMSInitiatingOccupancyOnly
> 14-:-XX:+UseG1GC
> 14-:-XX:G1ReservePercent=25
> 14-:-XX:InitiatingHeapOccupancyPercent=30
> 
> ## JVM temporary directory
> -Djava.io.tmpdir=${ES_TMPDIR}
> 
> ## heap dumps
> 
> # generate a heap dump when an allocation from the Java heap fails
> # heap dumps are created in the working directory of the JVM
> -XX:+HeapDumpOnOutOfMemoryError
> 
> # specify an alternative path for heap dumps; ensure the directory exists and
> # has sufficient space
> -XX:HeapDumpPath=data
> 
> # specify an alternative path for JVM fatal error logs
> -XX:ErrorFile=logs/hs_err_pid%p.log
> 
> ## JDK 8 GC logging
> 8:-XX:+PrintGCDetails
> 8:-XX:+PrintGCDateStamps
> 8:-XX:+PrintTenuringDistribution
> 8:-XX:+PrintGCApplicationStoppedTime
> 8:-Xloggc:logs/gc.log
> 8:-XX:+UseGCLogFileRotation
> 8:-XX:NumberOfGCLogFiles=32
> 8:-XX:GCLogFileSize=64m
> 
> # JDK 9+ GC logging
> 9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
> 
> ```

Last Node:

> **JVM**
>
> ```auto
> ## JVM configuration
> 
> ################################################################
> ## IMPORTANT: JVM heap size
> ################################################################
> ##
> ## You should always set the min and max JVM heap
> ## size to the same value. For example, to set
> ## the heap to 4 GB, set:
> ##
> ## -Xms4g
> ## -Xmx4g
> ##
> ## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
> ## for more information
> ##
> ################################################################
> 
> # Xms represents the initial size of total heap space
> # Xmx represents the maximum size of total heap space
> 
> -Xms3g
> -Xmx3g
> 
> ################################################################
> ## Expert settings
> ################################################################
> ##
> ## All settings below this section are considered
> ## expert settings. Don't tamper with them unless
> ## you understand what you are doing
> ##
> ################################################################
> 
> ## GC configuration
> 8-13:-XX:+UseConcMarkSweepGC
> 8-13:-XX:CMSInitiatingOccupancyFraction=75
> 8-13:-XX:+UseCMSInitiatingOccupancyOnly
> 
> ## G1GC Configuration
> # NOTE: G1 GC is only supported on JDK version 10 or later
> # to use G1GC, uncomment the next two lines and update the version on the
> # following three lines to your version of the JDK
> # 10-13:-XX:-UseConcMarkSweepGC
> # 10-13:-XX:-UseCMSInitiatingOccupancyOnly
> 14-:-XX:+UseG1GC
> 14-:-XX:G1ReservePercent=25
> 14-:-XX:InitiatingHeapOccupancyPercent=30
> 
> ## JVM temporary directory
> -Djava.io.tmpdir=${ES_TMPDIR}
> 
> ## heap dumps
> 
> # generate a heap dump when an allocation from the Java heap fails
> # heap dumps are created in the working directory of the JVM
> -XX:+HeapDumpOnOutOfMemoryError
> 
> # specify an alternative path for heap dumps; ensure the directory exists and
> # has sufficient space
> -XX:HeapDumpPath=data
> 
> # specify an alternative path for JVM fatal error logs
> -XX:ErrorFile=logs/hs_err_pid%p.log
> 
> ## JDK 8 GC logging
> 8:-XX:+PrintGCDetails
> 8:-XX:+PrintGCDateStamps
> 8:-XX:+PrintTenuringDistribution
> 8:-XX:+PrintGCApplicationStoppedTime
> 8:-Xloggc:logs/gc.log
> 8:-XX:+UseGCLogFileRotation
> 8:-XX:NumberOfGCLogFiles=32
> 8:-XX:GCLogFileSize=64m
> 
> # JDK 9+ GC logging
> 9-:-Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m
> 
> ```

As far as I know I haven't changed anything other than XMS and XMG

**EDIT**  
When zoomed in the collector looks pretty normal. Maybe it just **looks bad because** of the **aggregation**?

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/d/6d9e00b415fa43f14652ac7c4af30259be5bb65d.png)

Zooming in:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/4/d/4df8f8676c2edb94afa848c83be847f514d34fdc.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/2/6233c8a0c5a5b6b1219ffe7da89fe657555b0967.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/d/3d0f766449478eaffea4681df76a5d1dc480f965.png)

* * *

What I noticed is that the **collector** on the two nodes now **collects more often**. Is this a problem?  
10min of collecting before changes on the faulty node:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/2/f24592607bb77105cae4ef7800459409dc58c91e.png)  
10min now:  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/7/775ba9dc948b29935dfe306deaeaf245b31219e5.png)

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [July 21, 2020, 11:45pm UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/11 "2020-07-21T23:45:01Z")

</div>

That looks like pretty typical garbage collection to me. Is it causing actual problems?

---

<div class="post-metadata">

**Author:** ![defalt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/defalt/32/71379_2.png) [@defalt](https://discuss.elastic.co/u/defalt)\
**Post date:** [July 23, 2020, 6:30am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/12 "2020-07-23T06:30:21Z")

</div>

As for what I can see it doesn't cause problems. It was just weird that the cycle changed so drastically only because I changed the Java version on one node.  
Thanks for your info.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 20, 2020, 6:30am UTC](https://discuss.elastic.co/t/used-heap-behaves-wired-jvm-heapsize-problems/241871/13 "2020-08-20T06:30:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
