# APM Java agent v1.27.0+ instrumention errors on modulepath **URL:** \ **Category:** APM\ **Tags:** java\ **Created:** [December 24, 2021, 8:23am UTC](https://discuss.elastic.co/t/apm-java-agent-v1-27-0-instrumention-errors-on-modulepath/292877 "2021-12-24T08:23:07Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![duponter](https://avatars.discourse-cdn.com/v4/letter/d/4bbf92/32.png) [@duponter](https://discuss.elastic.co/u/duponter)\ **Post date:** [December 24, 2021, 8:23am UTC](https://discuss.elastic.co/t/apm-java-agent-v1-27-0-instrumention-errors-on-modulepath/292877/1 "2021-12-24T08:23:07Z")
Currently, APM Java agent v1.26.0 is attached successfully to a **Spring Boot 2.6 application (JDK 17)**. This application has enabled Java modules and is therefore **running on the modulepath**. Although the APM Java Agent is not impacted to the Log4J vulnerability, we want to upgrade to the latest version (v1.28.3 atm). During start up, following exceptions are thrown abundantly: ```auto c.e.a.a.b.b.ErrorLoggingListener : Error on transformation org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler java.lang.AbstractMethodError: Receiver class co.elastic.apm.agent.bci.bytebuddy.MinimumClassFileVersionValidator does not define or inherit an implementation of the resolved method 'abstract net.bytebuddy.jar.asm.ClassVisitor wrap(net.bytebuddy.description.type.TypeDescription, net.bytebuddy.jar.asm.ClassVisitor, net.bytebuddy.implementation.Implementation$Context, net.bytebuddy.pool.TypePool, net.bytebuddy.description.field.FieldList, net.bytebuddy.description.method.MethodList, int, int)' of interface net.bytebuddy.asm.AsmVisitorWrapper. at net.bytebuddy@1.11.22/net.bytebuddy.asm.AsmVisitorWrapper$Compound.wrap(AsmVisitorWrapper.java:743) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining$WithFullProcessing$RedefinitionClassVisitor.visit(TypeWriter.java:4867) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:569) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.jar.asm.ClassReader.accept(ClassReader.java:424) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.dynamic.scaffold.TypeWriter$Default$ForInlining.create(TypeWriter.java:3918) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.dynamic.scaffold.TypeWriter$Default.make(TypeWriter.java:2192) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.dynamic.scaffold.inline.RedefinitionDynamicTypeBuilder.make(RedefinitionDynamicTypeBuilder.java:224) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:11362) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11298) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1700(AgentBuilder.java:11016) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:11763) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:11702) ~[byte-buddy-1.11.22.jar:na] at java.base/java.security.AccessController.doPrivileged(AccessController.java:399) ~[na:na] at net.bytebuddy@1.11.22/net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doPrivileged(AgentBuilder.java) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy@1.11.22/net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:11240) ~[byte-buddy-1.11.22.jar:na] at net.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[na:na] at java.instrument/sun.instrument.TransformerManager.transform(TransformerManager.java:188) ~[na:na] at java.instrument/sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:541) ~[na:na] at java.base/java.lang.ClassLoader.defineClass1(Native Method) ~[na:na] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1012) ~[na:na] at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1090) ~[na:na] at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:182) ~[na:na] at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:821) ~[na:na] at java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(BuiltinClassLoader.java:741) ~[na:na] at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:665) ~[na:na] at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639) ~[na:na] at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[na:na] at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) ~[na:na] ``` When reverting to the v1.26 version, the agent attaches normally. When running the same application on the classpath, the agent v1.28.3 attaches normally. After more research, we suspect the error appears after following change has been applied: > > > \## What does this PR do? > > Since version 1.23.0, the attacher jar does not embe…ds bytebuddy dependency anymore. > > This becomes an issue when a different version of bytebuddy is on the application classpath (for example through an hibernate dependency) as it makes the attach fail with a similar stack trace: > > \`\`\` > Exception in thread "main" java.lang.reflect.InvocationTargetException > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48) > at org.springframework.boot.loader.Launcher.launch(Launcher.java:87) > at org.springframework.boot.loader.Launcher.launch(Launcher.java:50) > at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51) > Caused by: java.lang.NoClassDefFoundError: net/bytebuddy/agent/ByteBuddyAgent$AttachmentProvider$ForEmulatedAttachment > at co.elastic.apm.attach.ElasticAttachmentProvider.init(ElasticAttachmentProvider.java:39) > at co.elastic.apm.attach.ElasticAttachmentProvider.get(ElasticAttachmentProvider.java:60) > at co.elastic.apm.attach.ElasticApmAttacher.attach(ElasticApmAttacher.java:162) > at co.elastic.apm.attach.ElasticApmAttacher.attach(ElasticApmAttacher.java:148) > at co.elastic.apm.attach.ElasticApmAttacher.attach(ElasticApmAttacher.java:110) > at co.elastic.apm.attach.ElasticApmAttacher.attach(ElasticApmAttacher.java:63) > at org.springframework.samples.petclinic.PetClinicApplication.main(PetClinicApplication.java:34) > ... 8 more > Caused by: java.lang.ClassNotFoundException: net.bytebuddy.agent.ByteBuddyAgent$AttachmentProvider$ForEmulatedAttachment > at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) > at org.springframework.boot.loader.LaunchedURLClassLoader.loadClass(LaunchedURLClassLoader.java:93) > at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) > ... 15 morev > \`\`\` > > This previous shading change has been introduced in version 1.23.0 through \[this commit\](https://github.com/elastic/apm-agent-java/pull/1667/commits/d6a262f02f7e07b8be55750a59636fa7c60818e1) which is part of \[PR #1667\](https://github.com/elastic/apm-agent-java/pull/1667) > > Since \`attacher-cli\` had a few minor dependencies on bytebuddy and it was provided through a transitive dependency of \`attacher\` which now relocates bytebuddy, it was necessary to move few parts to \`attacher\` for convenience and to allow removing dependencies from \`attacher-cli\` to bytebuddy. > > \## Checklist > > \- \[x\] This is a bugfix > - \[x\] I have updated \[CHANGELOG.asciidoc\](https://github.com/elastic/apm-agent-java/blob/master/CHANGELOG.asciidoc) > - \[x\] I have added tests that would fail without this fix > - \[x\] test on Windows We could reproduce this error when generating a new Spring Boot application (with only Spring Data Jpa and Spring MVC) and converting it to the module path. For now, we are sticking to the latest v1.26.x Agent. ---
**Author:** ![Jack\_Shirazi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_shirazi/32/91641_2.png) [@Jack\_Shirazi](https://discuss.elastic.co/u/Jack_Shirazi)\ **Post date:** [December 24, 2021, 10:07am UTC](https://discuss.elastic.co/t/apm-java-agent-v1-27-0-instrumention-errors-on-modulepath/292877/2 "2021-12-24T10:07:00Z")
Thanks. 1.26.1 is available as you note, exactly because of the large architectural change that you see in 1.27. What would be ideal for us is if you can provide an isolated example which triggers that bug. ---
**Author:** ![duponter](https://avatars.discourse-cdn.com/v4/letter/d/4bbf92/32.png) [@duponter](https://discuss.elastic.co/u/duponter)\ **Post date:** [December 24, 2021, 11:53am UTC](https://discuss.elastic.co/t/apm-java-agent-v1-27-0-instrumention-errors-on-modulepath/292877/3 "2021-12-24T11:53:53Z")
As I mentioned in the topic, we could reproduce the bug with a newly generated Spring Boot application. The following link refers to an apm-modules.zip file which contains a single module Maven project with a simple Spring Boot 2.6 application. > **[apm-modules.zip](https://drive.google.com/file/d/1qCv4QGG4RRwHqgjo9qKAa4wShPczgeSp/view?usp=sharing)** > > Google Drive file. It contains a readme.md with instructions to run and trigger the bug. It does not include the APM agent jars (to keep the zip as small as possible). You’ll have to add them manually in the project’s root directory. Hopefully, it helps you any further. Thanks for looking into this ---
**Author:** ![Jack\_Shirazi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jack_shirazi/32/91641_2.png) [@Jack\_Shirazi](https://discuss.elastic.co/u/Jack_Shirazi)\ **Post date:** [December 24, 2021, 3:10pm UTC](https://discuss.elastic.co/t/apm-java-agent-v1-27-0-instrumention-errors-on-modulepath/292877/4 "2021-12-24T15:10:12Z")
Thank you, that's very useful. It looks like it'll take us some time (especially given holiday period) to resolve this ---
**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\ **Post date:** [January 4, 2022, 9:04am UTC](https://discuss.elastic.co/t/apm-java-agent-v1-27-0-instrumention-errors-on-modulepath/292877/5 "2022-01-04T09:04:49Z")
We also have a github issue ([Fix modularized applications having a dependency version conflict · Issue #2373 · elastic/apm-agent-java · GitHub](https://github.com/elastic/apm-agent-java/issues/2373)) to keep track of this bug. You can subscribe to it to get notified when we are making progress on it. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 25, 2022, 5:05am UTC](https://discuss.elastic.co/t/apm-java-agent-v1-27-0-instrumention-errors-on-modulepath/292877/6 "2022-01-25T05:05:06Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.