# Wrong monitoring statistics - jvm heap size

**URL:** <https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655>\
**Category:** Logstash\
**Created:** [March 15, 2017, 8:15am UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655 "2017-03-15T08:15:12Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)\
**Post date:** [March 15, 2017, 8:15am UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/1 "2017-03-15T08:15:12Z")

</div>

Hi All,

I just compared the jvm heap usage from kibana and visualvm and realized, that Kibana shows incorrect values.

```
LS: v.5.2.2
LS_HEAP_SIZE=512m

```

Charts:

[![](https://s22.postimg.org/618cydbo1/LS-monitoring-vgc.png) ](https://s22.postimg.org/618cydbo1/LS-monitoring-vgc.png)[![](https://s22.postimg.org/n0cpzvxht/LS-monitoring-kibana.png) ](https://s22.postimg.org/n0cpzvxht/LS-monitoring-kibana.png)

What's wrong the monitoring data from LS?  
Thanks!

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 15, 2017, 2:30pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/2 "2017-03-15T14:30:18Z")

</div>

Have you checked the `ps` tree to see what the actual Xms and Xmx are set to? `LS_HEAP_SIZE` isn't used anymore. Heap is set in the `jvm.options` file now.

---

<div class="post-metadata">

**Author:** ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)\
**Post date:** [March 15, 2017, 3:18pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/3 "2017-03-15T15:18:21Z")

</div>

yes, i checked the process list with the Process explorer (it's on windows 😊)  
the heap-size variable works for some reason:

`"c:\progra~1\java\jdk\jre\bin\java.exe" "-Djdk.home=c:\progra~1\java\jdk" "-Djruby.home=C:\programms\logstash\vendor\jruby" "-Djruby.script=jruby" "-Djruby.shell=cmd.exe" "-Djffi.boot.library.path=C:\programms\logstash\vendor\jruby\lib\jni;C:\programms\logstash\vendor\jruby\lib\jni\i386-Windows;C:\programms\logstash\vendor\jruby\lib\jni\x86_64-Windows" "-Xss2048k" "-Dsun.java.command=org.jruby.Main" "-Djava.class.path=" "-Xbootclasspath/a:C:\programms\logstash\vendor\jruby\lib\jruby.jar" "-Xmx512m" "-XX:+UseParNewGC" "-XX:+UseConcMarkSweepGC" "-XX:+CMSParallelRemarkEnabled" "-XX:SurvivorRatio=8" "-XX:MaxTenuringThreshold=1" "-XX:CMSInitiatingOccupancyFraction=75" "-XX:+UseCMSInitiatingOccupancyOnly" "-XX:+HeapDumpOnOutOfMemoryError" "-XX:HeapDumpPath="C:\programms\logstash/heapdump.hprof"" org/jruby/Main "C:\programms\logstash\lib\bootstrap\environment.rb" "logstash\runner.rb" "-f" "C:\projects\logstash.conf" "--path.settings" "C:/etc/logstash"`

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 15, 2017, 3:32pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/4 "2017-03-15T15:32:16Z")

</div>

Ah, Windows. The `jvm.options` file doesn't work in Windows. If `LS_HEAP_SIZE` works for now, great. It might go away in the future. In that case, you should instead edit `setup.bat` to use the JVM settings you want.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 15, 2017, 3:45pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/5 "2017-03-15T15:45:04Z")

</div>

To address your concern, the blue line (marked as "UsedHeap") is tracking your heap usage. The pink line (marked as "MaxHeap") is not quite 2x your set heap size. Under some conditions, the Xmx may be exceeded by the JVM, and this arrangement helps us to show what's going on in such a case. As you will note, though, the blue line is staying comfortably below the 512m mark, and is doing garbage collection _very_ regularly. Perhaps a bit too rapidly, in fact. I would suggest doubling your heap size to 1g to see if it helps. Your event processing sawtooth pattern will be more even with fewer garbage collection events from Logstash.

---

<div class="post-metadata">

**Author:** ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)\
**Post date:** [March 15, 2017, 4:29pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/6 "2017-03-15T16:29:57Z")

</div>

but if you compare the heap usage size - the visual VM shows, that the heap size stays below 220m, and the LS statistics shows almost doubled values (around 400m).  
To be honest, I'd trust the visualVM data

To say nothing of the empty CPU usage chart...

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 15, 2017, 4:37pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/7 "2017-03-15T16:37:34Z")

</div>

We'll investigate. Truthfully, it may have something to do with metric collection on windows, and the libraries being used. On my own (linux) system, I see the same behavior with the JVM, and the GC is triggered _exactly_ at the Xmx threshold.

---

<div class="post-metadata">

**Author:** ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)\
**Post date:** [March 15, 2017, 4:38pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/8 "2017-03-15T16:38:48Z")

</div>

ah, ok, it's good to know, that on Unix it works as expected.  
thank you!

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 15, 2017, 4:42pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/9 "2017-03-15T16:42:41Z")

</div>

Are you using a container, or VM perhaps? Or is this running in a single Windows instance?

---

<div class="post-metadata">

**Author:** ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)\
**Post date:** [March 15, 2017, 4:43pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/10 "2017-03-15T16:43:46Z")

</div>

no VM, it's a "normal" host system - win10

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 15, 2017, 4:45pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/11 "2017-03-15T16:45:45Z")

</div>

I just spoke with a team member who has been addressing this. It's a [known issue with Windows](https://github.com/elastic/logstash/issues/6653), and he said he'd get a fix out soon.

---

<div class="post-metadata">

**Author:** ![theuntergeek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/theuntergeek/32/44961_2.png) [@theuntergeek](https://discuss.elastic.co/u/theuntergeek)\
**Post date:** [March 15, 2017, 4:47pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/12 "2017-03-15T16:47:34Z")

</div>

Also, apparently math is fun: [https://github.com/elastic/logstash/issues/6608](https://github.com/elastic/logstash/issues/6608)

---

<div class="post-metadata">

**Author:** ![jetnet](https://avatars.discourse-cdn.com/v4/letter/j/a87d85/32.png) [@jetnet](https://discuss.elastic.co/u/jetnet)\
**Post date:** [March 15, 2017, 4:50pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/13 "2017-03-15T16:50:05Z")

</div>

ok, I'll start my research with github next time 😉  
waiting for v.5.4  
Thank you again for you time!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2017, 4:50pm UTC](https://discuss.elastic.co/t/wrong-monitoring-statistics-jvm-heap-size/78655/14 "2017-04-12T16:50:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
