Not able to add JAVA APM agent as dependency in spring boot project

Elastic Observability APM
1

I want to do distributed tracking using Java APM.
Initially, we use Java agent as manual setup with -javaagent flag and we get tracing data using this. Then we deploy this on our test server and its working fine.
Now, to take this on production server, i try to add Java APM as dependency but its throwing error as follows:

[INFO] --- spring-boot-maven-plugin:2.1.6.RELEASE:run (default-cli) @ demo ---
[INFO] Attaching agents: 
2019-07-29 10:41:47.841 [Attach Listener] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.7.0 as first on Java 1.8.0_212 (Oracle Corporation) Linux 4.18.0-25-generic
2019-07-29 10:41:47.841 [Attach Listener] WARN co.elastic.apm.agent.configuration.StartupInfo - To enable all features and to increase startup times, please configure application_packages
2019-07-29 10:41:47.925 [apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: { "build_date": "2019-06-20T14:39:23Z", "build_sha": "9a099b63c53eac8c55707df96193143ec66337e9", "version": "7.2.0"}
Exception in thread "restartedMain" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: java.lang.IllegalStateException: Error during attachment using: co.elastic.apm.attach.bytebuddy.agent.ByteBuddyAgent$AttachmentProvider$Compound@867f9f9
at co.elastic.apm.attach.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:427)
at co.elastic.apm.attach.bytebuddy.agent.ByteBuddyAgent.attach(ByteBuddyAgent.java:298)
at co.elastic.apm.attach.bytebuddy.agent.ByteBuddyAgent.attach(ByteBuddyAgent.java:273)
at co.elastic.apm.attach.ElasticApmAttacher.attach(ElasticApmAttacher.java:84)
at co.elastic.apm.attach.ElasticApmAttacher.attach(ElasticApmAttacher.java:55)
at com.example.demo.DemoApplication.main(DemoApplication.java:11)
... 5 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at co.elastic.apm.attach.bytebuddy.agent.Attacher.install(Attacher.java:99)
at co.elastic.apm.attach.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:422)
... 10 more
Caused by: java.lang.UnsatisfiedLinkError: Native Library /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/libattach.so already loaded in another classloader
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1907)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1845)
at java.lang.Runtime.loadLibrary0(Runtime.java:870)
at java.lang.System.loadLibrary(System.java:1122)
at sun.tools.attach.LinuxVirtualMachine.(LinuxVirtualMachine.java:342)
at sun.tools.attach.LinuxAttachProvider.attachVirtualMachine(LinuxAttachProvider.java:63)
at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:208)
... 16 more

We are using Java APM version :1.7.0
Code in pom.xml:

co.elastic.apm
apm-agent-attach
1.7.0

2

That most likely means that you are using another library which has already loaded the libattach native library. The JVM restricts native libraries to be loaded from only one classloader.

Are you using libraries like EhCache's size-of agent?

However, it seems you are calling ElasticApmAttacher.attach in the first line of your main method (DemoApplication.java:11)? At that point, no other library would have the chance to load the libattach library. My guess is that because of spring-boot's restarting behavior (Exception in thread "restartedMain"), ElasticApmAttacher.attach is called twice and from two different class loaders. However, I have never seen that becoming a problem before, so I'm a bit puzzled why this happens only to you. It could be related to the next question: Which JVM version are you using? I would guess 8? If you have to possibility to try with 9 or newer, please try that.

I'll add a guard so that attachment won't be performed twice, that should fix it.

EDIT: just noticed that you are using java-8-openjdk-amd64 which confirms my theory. I have also tested with JDK 8 but couldn't reproduce. Anyway, my planned fix would help here. I'll update with the GitHub PR

3

Here is the PR: https://github.com/elastic/apm-agent-java/pull/764

4

In the mean time, you can do this:

if (!Boolean.getBoolean("ElasticApm.attached")) {
    ElasticApmAttacher.attach();
}
5

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.