Java APM1.41.0 JVM used is Zero

[zt9788]

1.38.0 can catch the jvm used
Programs typically use around 200-300M of heap memory
Non-Heap memory is 260~320M in 1.38.0 befor

JDK:
JDK17.0.8

Kibana version:
7.13.2
Elasticsearch version:
7.13.2
APM Server version:
7.13.2

APM Agent language and version:
1.41.0

[zt9788]

Hi, i try the new 1.41.1 but the displayed data is still incorrect

It seems that the data is exactly 10 times different

[zt9788]

the bug is the same with Bug: APM Java - Springboot, v1.39.0+ - No JVM Metrics Found - #12 by Jonas_Kunz

[Sylvain_Juge]

Hi,

I've tried to reproduce with the same stack version and Java agent version and it seems that metrics are properly captured, however, the rounding in the UI could definitely cause some confusion here as overlay is shown with the biggest unit (here GB).

• you likely have set a limit to 1.3gb with -Xmx, thus the overlay uses gigabyte as units.
• the application currently have a very small usage of heap (less than 100mb), thus it gets rounded to 0.0 GB in the overlay.

Could you try to let the application run for a while with some activity so we can see how does the metrics look like when the heap usage goes over 100mb ?
Alternatively, could you try to remove the -Xmx argument from your JVM ? with that the heap usage should be shown in megabytes instead of gigabytes, hence there shouldn't be any rounding here.

[zt9788]

I try another program.

This does not use the disable_metrics parameter, XMX3250
And the Programe is running more then one hour(used about 700M memory+-)

---

this is use the -Delastic.apm.disable_metrics=jvm.fd.*,jvm.memory.non_heap.pool.*
XMX=3086, used 500M~2G

JVM

-Xms1024m -Xmx3086m
-XX:+UseContainerSupport 
-XX:MinRAMPercentage=50.0 -XX:MaxRAMPercentage=80.0 
-Delastic.apm.log_level=ERROR 
-Delastic.apm.disable_metrics=jvm.fd.*,jvm.memory.non_heap.pool.* 
-Delastic.apm.service_node_name=xxx
-Delastic.apm.service_name=xxxxx
-Delastic.apm.transaction_sample_rate=0.8

I deploy it in a container, and in order to prevent memory overflow, XMX is calculated. In the production environment, I am currently afraid to delete XMX

There also can not display jvm used

this is apm 1.38.0

the data is same with the jmap

---

I will run it for a day first to see if there will be any changes when the usage exceeds 1G

[zt9788]

I search the apm-metric index
when apm is 1.38.0
the promram real used about 1.9G

"non_heap" : {
      "committed" : 3.74669312E8,
      "max" : 9.77272832E8,
      "used" : 3.663926E8
    },
    "heap" : {
      "committed" : 2.76824064E9,
      "max" : 3.235905536E9,
      "used" : 1.956814336E9
    }

when apm is 1.41.x
the program real used 1.2G used = 1078312208 (1028.358657836914MB)
but data is 3.45344E7 used

    "non_heap" : {
      "committed" : 3.4930688E7,
      "max" : 9.89855744E8,
      "used" : 3.1833088E7
    },
    "heap" : {
      "committed" : 1.077936128E9,
      "max" : 3.235905536E9,
      "used" : 3.45344E7
    }

[Sylvain_Juge]

Hi,

I've managed to reproduce it, and that's due to a regression introduced in 1.39.0 which hasn't been reported yet, so congrats for catching this !

I've opened Fix JVM memory metrics regression by SylvainJuge · Pull Request #3279 · elastic/apm-agent-java · GitHub to fix it and it's already merged thus will be included in the next release.

So in your case you should either:

• keep using 1.38.0 until next release is available
• use the snapshot that includes the bugfix: GitHub - elastic/apm-agent-java: Elastic APM Java Agent

[zt9788]

Thank you very much. Using the snapshot version, the issue has been corrected