# Bug: APM Java - Springboot, v1.39.0+ - No JVM Metrics Found **URL:** \ **Category:** APM\ **Tags:** java\ **Created:** [July 24, 2023, 1:47pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104 "2023-07-24T13:47:20Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [July 24, 2023, 1:47pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/1 "2023-07-24T13:47:20Z")
After reviewing the [following post](https://discuss.elastic.co/t/jvm-metrics-failing-on-latest-java17-openj9-with-the-java-agent/337863) and appropriate [GitHub issue](https://github.com/elastic/apm-agent-java/issues/3224), utilizing the Java APM agent on versions 1.39.0+ (including 1.40.0), JVM Metrics fail to display in the Metrics tab of APM. Below is an example of me swapping the agent versions to show the drop-off: ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/9/e983d319f322d1ad70d7d31ef102a0c9f2018aa3.png) After reviewing APM log levels, no error logs are thrown, and debug logs are very verbose, making tacking this source issue down difficult. I can post debug logs on request. Versions: Java - openjdk version "11.0.19" 2023-04-18 OpenJDK Runtime Environment (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1) Springboot versions: 2.5.1, 3.x.x OpenJDK 64-Bit Server VM (build 11.0.19+7-post-Ubuntu-0ubuntu122.04.1, mixed mode, sharing) APM - 1.39.0, 1.40.0 ---
**Author:** ![Jonas\_Kunz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonas_kunz/32/110740_2.png) [@Jonas\_Kunz](https://discuss.elastic.co/u/Jonas_Kunz)\ **Post date:** [July 25, 2023, 9:28am UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/2 "2023-07-25T09:28:16Z")
Unfortunately we will need the debug-level agent logs here to help you further. You can use [GitHub Gists](https://gist.github.com/) to upload the logs. I think the linked issue should be unrelated, as it is specific to IBM Semeru JDK17. ---
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [July 25, 2023, 12:22pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/3 "2023-07-25T12:22:58Z")
Great, here you go: > ---
**Author:** ![Jonas\_Kunz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonas_kunz/32/110740_2.png) [@Jonas\_Kunz](https://discuss.elastic.co/u/Jonas_Kunz)\ **Post date:** [July 26, 2023, 7:52am UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/4 "2023-07-26T07:52:22Z")
The logs indicate that metrics are actually being sent: ```auto 2023-07-25 08:45:06,685 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 1) 2023-07-25 08:45:06,685 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 2) 2023-07-25 08:45:06,685 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 3) 2023-07-25 08:45:06,685 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 4) 2023-07-25 08:45:06,685 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 5) 2023-07-25 08:45:06,686 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 6) 2023-07-25 08:45:06,686 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 7) 2023-07-25 08:45:06,686 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 8) 2023-07-25 08:45:06,686 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICSET_JSON_WRITER event (sequence 9) ``` To verify that those are actually the JVM metrics and that they make it to elasticsearch please do the following: - Start your application with apm agent 1.39.0 and leave it running for a while - In Kibana got to discover and - Select a time range of at least a few minutes within which your application was runnign with agent 1.39.0 - Select the `APM` dataview - Enter the following filter: `service.name:"javaPyro" and processor.event :"metric" and "system.cpu.total.norm.pct" : *` You should then get some documents containing the CPU usage if the metrics made it from the agent to elasticsearch. ---
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [July 26, 2023, 12:54pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/5 "2023-07-26T12:54:27Z")
Performed the above, there are no docs per the query. I have attached a query searching for any metrics, and then system.cpu docs to show the difference. ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/c/dc21c7ebba7a42289bba945d8cdaa3d6d8d7419e.png) ---
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [July 26, 2023, 12:55pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/6 "2023-07-26T12:55:02Z")
Here is the full query's result ![image](https://us1.discourse-cdn.com/elastic/original/3X/2/4/2490b848933a84d616a1bcfc50b3b9b4ace4431f.png) ---
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [July 26, 2023, 1:13pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/7 "2023-07-26T13:13:46Z")
Here is a snippet of the server side logs at that time: > ---
**Author:** ![Jonas\_Kunz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonas_kunz/32/110740_2.png) [@Jonas\_Kunz](https://discuss.elastic.co/u/Jonas_Kunz)\ **Post date:** [July 26, 2023, 2:41pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/8 "2023-07-26T14:41:53Z")
This seems to be an APM-server issue related to some new JVM metrics which were added in the Java Agent 1.39.0. Which APM-server version are you using? ---
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [July 26, 2023, 2:58pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/9 "2023-07-26T14:58:32Z")
AM Server Version: 8.7.0 ---
**Author:** ![Jonas\_Kunz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonas_kunz/32/110740_2.png) [@Jonas\_Kunz](https://discuss.elastic.co/u/Jonas_Kunz)\ **Post date:** [July 27, 2023, 7:46am UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/10 "2023-07-27T07:46:18Z")
I've been able to reproduce the issue. The newly added JVM metrics are not correctly mapped by pre 8.9.0 APM server versions, causing the metricsets (which include other JVM metrics) to be dropped. I would suggest you to either - Upgrade your APM-server to 8.9.0 - Disable the "bad" metrics in the APM Agent via `disable_metrics=jvm.fd.*,jvm.memory.non_heap.pool.*` ---
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [July 27, 2023, 12:10pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/11 "2023-07-27T12:10:24Z")
Awesome, thanks for looking into it Jonas ---
**Author:** ![Jonas\_Kunz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jonas_kunz/32/110740_2.png) [@Jonas\_Kunz](https://discuss.elastic.co/u/Jonas_Kunz)\ **Post date:** [July 27, 2023, 3:00pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/12 "2023-07-27T15:00:28Z")
Minor clarification: We didn't try reproducing the issue, but came up with an explanation. Unfortunately that explanation doesn't seem to be valid: I tried to actually reproduce the behaviour you are seeing with an 8.7.0 deployment and can't reproduce it. It looks like your deployment of the apm-package is somehow incorrect / corrupted. Nonetheless `disable_metrics=jvm.fd.*,jvm.memory.non_heap.pool.*` still should do the trick for you. ---
**Author:** ![zt9788](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/zt9788/32/113887_2.png) [@zt9788](https://discuss.elastic.co/u/zt9788)\ **Post date:** [August 8, 2023, 6:45am UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/13 "2023-08-08T06:45:11Z")
Hi, I set this `-Delastic.apm.disable_metrics=jvm.fd.*,jvm.memory.non_heap.pool.*` but JVM metrics also not found in apm7.13.2 ,just only upgrade apm-server? ---
**Author:** ![6fears7](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/6fears7/32/123799_2.png) [@6fears7](https://discuss.elastic.co/u/6fears7)\ **Post date:** [August 8, 2023, 12:10pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/14 "2023-08-08T12:10:44Z")
I tested a higher version of the APM server and the problem went away. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [September 5, 2023, 12:11pm UTC](https://discuss.elastic.co/t/bug-apm-java-springboot-v1-39-0-no-jvm-metrics-found/339104/15 "2023-09-05T12:11:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.