# APM Agent High CPU Load

**URL:** <https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655>\
**Category:** APM\
**Tags:** java\
**Created:** [May 28, 2020, 5:21am UTC](https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655 "2020-05-28T05:21:30Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![niklaskhf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niklaskhf/32/69245_2.png) [@niklaskhf](https://discuss.elastic.co/u/niklaskhf)\
**Post date:** [May 28, 2020, 5:21am UTC](https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655/1 "2020-05-28T05:21:30Z")

</div>

I am trying to instrument a SpringBoot application with the Java APM agent.  
The SpringBoot application runs in a container based on the arm64v8/adoptopenjdk image. The APM agent is attached with the -javaagent method in the Docker CMD. SERVER\_URLS, SERVICE\_NAME, ENVIRONMENT, APPLICATION\_PACKAGES, LOG\_FILE and LOG\_LEVEL are configured through environment variables. Additionally, I disabled all instrumentations by adding all possible options to -Delastic.apm.disable\_instrumentations.  
While the startup of the SpringBoot usually takes about 60ms, attaching the APM agent causes the startup to take over 50s. This is caused by an extremely high CPU usage of 100%, which starts as soon as the APM agent starts.

Is this a problem with the agent running on ARM?

Other Go services that run on the same node do not experience any of these CPU issues.

**Kibana version** : 7.7

**Elasticsearch version** : 7.7

**APM Server version** : 7.7

**APM Agent language and version** : Java, 1.16

**Browser version** : Irrelevant

**Original install method (e.g. download page, yum, deb, from source, etc.) and version**: [https://search.maven.org/remotecontent?filepath=co/elastic/apm/elastic-apm-agent/1.16.0/elastic-apm-agent-1.16.0.jar](https://search.maven.org/remotecontent?filepath=co/elastic/apm/elastic-apm-agent/1.16.0/elastic-apm-agent-1.16.0.jar)

**Fresh install or upgraded from other version?** Fresh install

**Is there anything special in your setup?** The Java SpringBoot application is running in an arm64 container.

**Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**:

**Steps to reproduce** :

1. Attach agent to SpringBoot application running in arm64v8/adoptopenjdk based container

**Errors in browser console (if relevant)**: None

**Provide logs and/or server output (if relevant)**: Logs: [https://gist.github.com/niklaskhf/b8fcd36b29d8d2a1e10a4338ff376435](https://gist.github.com/niklaskhf/b8fcd36b29d8d2a1e10a4338ff376435)

---

<div class="post-metadata">

**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\
**Post date:** [May 28, 2020, 7:06am UTC](https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655/2 "2020-05-28T07:06:15Z")

</div>

That's definitely strange and unexpected. Especially since you have disabled all instrumentations.  
According to the logs, the agent starts up in under one second.

Could you get some thread dumps or attach a profiler at the startup?

---

<div class="post-metadata">

**Author:** ![niklaskhf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niklaskhf/32/69245_2.png) [@niklaskhf](https://discuss.elastic.co/u/niklaskhf)\
**Post date:** [May 28, 2020, 7:47am UTC](https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655/3 "2020-05-28T07:47:14Z")

</div>

I took some thread dumps during various stages of the startup:

> <https://gist.github.com/niklaskhf/951aca429579e6bb3642008bd876fc56>
>
> There are more than three files. show original

After the startup eventually finishes, the CPU load goes down from 100% to ~20%, which is still higher than without the agent.

Edit: These are with the instrumentation enabled, resulting in a startup time of about 140s.

---

<div class="post-metadata">

**Author:** ![felixbarny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/felixbarny/32/27341_2.png) [@felixbarny](https://discuss.elastic.co/u/felixbarny)\
**Post date:** [May 28, 2020, 8:13am UTC](https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655/4 "2020-05-28T08:13:04Z")

</div>

> [@niklaskhf](#):
>
> After the startup eventually finishes, the CPU load goes down from 100% to ~20%, which is still higher than without the agent.

What's the normal baseline?

I couldn't spot any obvious culprit. Maybe park/unpark is especially expensive with ARM and Java 8?

What's the CPU you're running it on? Is there a chance you could try with a more recent Java version?

---

<div class="post-metadata">

**Author:** ![niklaskhf](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/niklaskhf/32/69245_2.png) [@niklaskhf](https://discuss.elastic.co/u/niklaskhf)\
**Post date:** [May 28, 2020, 8:29am UTC](https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655/5 "2020-05-28T08:29:39Z")

</div>

The usual idle load is ~8%. I will try to run this setup on different instance type / Java version configurations and report back if I find anything wrong with the ARM / Java 8 version in particular. It's running on an a1.medium AWS instance. I have a feeling that upgrading the instance type and CPU might already help with this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 18, 2020, 4:36am UTC](https://discuss.elastic.co/t/apm-agent-high-cpu-load/234655/6 "2020-06-18T04:36:24Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
