# Unable to dynamically update instrumentation=true **URL:** \ **Category:** APM\ **Tags:** java\ **Created:** [May 14, 2020, 12:44pm UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645 "2020-05-14T12:44:24Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![Ananda\_Babu](https://avatars.discourse-cdn.com/v4/letter/a/f475e1/32.png) [@Ananda\_Babu](https://discuss.elastic.co/u/Ananda_Babu)\ **Post date:** [May 14, 2020, 12:44pm UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645/1 "2020-05-14T12:44:24Z")
**Kibana version** : 7.6.2 **Elasticsearch version** : 7.6.2 **APM Server version** : 7.6.2 **APM Agent language and version** : Java - elastic-apm-agent-1.15.0.jar **Original install method and version** : APT repository **Fresh install or upgraded from other version?** Fresh install **Is there anything special in your setup?** None **Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**: Hi I'm trying to start my tomcat with recording and instrumentation set to false, and then on demand i'm trying to set them to true during debug sessions. The configs are kept in the elasticapm.properties file in the same directory as the jar file (/opt/). While setting recording=true, I could see the JVM metrics are sent to apm-server thus displayed in Kibana. However setting instrumentation=true, I couldn't get any data captured (the application is constantly accessed by a script to simulate the traffic for the demo) I couldn't find any error in my apm-agent.log file (set via log\_file=/var/log/tomcat9/apm-agent.log parameter). But the syslog from tomcat9 throws the below exception. `java.lang.ClassNotFoundException: co.elastic.apm.agent.shaded.bytebuddy.agent.Installer` I've attached the logs for further reference. The same applies to when you try to set the instrumentation to false from true. The data is continued to sent to apm-server. **Steps to reproduce** : 1. Start the agent with recording & instrumentation set to false 2. After starting the tomcat9, change the instrumentation to true in the elaspticapm.properties 3. watch the syslog (tail -f /var/log/syslog | grep apm) **Provide logs and/or server output (if relevant)**: ```auto #tail -f /var/log/syslog | grep apm May 14 12:12:15 tomcat tomcat9[6687]: #011at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default.patchOnByteBuddyAgent(AgentBuilder.java:9435) May 14 12:12:15 tomcat tomcat9[6687]: #011at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$Delegator.patchOnByteBuddyAgent(AgentBuilder.java:11007) May 14 12:12:15 tomcat tomcat9[6687]: #011at co.elastic.apm.agent.bci.ElasticApmAgent.doReInitInstrumentation(ElasticApmAgent.java:196) May 14 12:12:15 tomcat tomcat9[6687]: #011at co.elastic.apm.agent.bci.ElasticApmAgent$3.run(ElasticApmAgent.java:183) May 14 12:12:15 tomcat tomcat9[6687]: Caused by: java.lang.ClassNotFoundException: co.elastic.apm.agent.shaded.bytebuddy.agent.Installer May 14 12:12:15 tomcat tomcat9[6687]: #011at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default.patchOnByteBuddyAgent(AgentBuilder.java:9429) ``` apm-agent.log = [https://gist.github.com/Anandababu/dc3b04f4284157c0f240acc55934c9ec](https://gist.github.com/Anandababu/dc3b04f4284157c0f240acc55934c9ec) ---
**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\ **Post date:** [May 14, 2020, 12:58pm UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645/2 "2020-05-14T12:58:15Z")
Hi and welcome to the forum! How did you install the agent? ---
**Author:** ![Ananda\_Babu](https://avatars.discourse-cdn.com/v4/letter/a/f475e1/32.png) [@Ananda\_Babu](https://discuss.elastic.co/u/Ananda_Babu)\ **Post date:** [May 14, 2020, 1:01pm UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645/3 "2020-05-14T13:01:35Z")
Apologies , should have included the info.. The apm-agent is attched via the -javaagent flag in the setenv.sh file in the catalina\_base like below ```auto export CATALINA_OPTS="$CATALINA_OPTS -javaagent:/opt/elastic-apm-agent-1.15.0.jar" export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.service_name=my-application" export CATALINA_OPTS="$CATALINA_OPTS -Delastic.apm.server_url=http://apm.com:8200" ``` The dashboards are all fine in displaying the metrics when tomcat9 started with instrumentation=true. Its just that not possible to update them dynamically. ---
**Author:** ![Ananda\_Babu](https://avatars.discourse-cdn.com/v4/letter/a/f475e1/32.png) [@Ananda\_Babu](https://discuss.elastic.co/u/Ananda_Babu)\ **Post date:** [May 14, 2020, 1:57pm UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645/4 "2020-05-14T13:57:29Z")
```auto May 14 13:56:43 tomcat tomcat9[7665]: java.lang.IllegalStateException: The Byte Buddy agent is not installed or not accessible May 14 13:56:43 tomcat tomcat9[7665]: #011at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default.patchOnByteBuddyAgent(AgentBuilder.java:9435) May 14 13:56:43 tomcat tomcat9[7665]: #011at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$Delegator.patchOnByteBuddyAgent(AgentBuilder.java:11007) May 14 13:56:43 tomcat tomcat9[7665]: #011at co.elastic.apm.agent.bci.ElasticApmAgent.doReInitInstrumentation(ElasticApmAgent.java:196) May 14 13:56:43 tomcat tomcat9[7665]: #011at co.elastic.apm.agent.bci.ElasticApmAgent$3.run(ElasticApmAgent.java:183) May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/java.lang.Thread.run(Thread.java:834) May 14 13:56:43 tomcat tomcat9[7665]: Caused by: java.lang.ClassNotFoundException: co.elastic.apm.agent.shaded.bytebuddy.agent.Installer May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) May 14 13:56:43 tomcat tomcat9[7665]: #011at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) May 14 13:56:43 tomcat tomcat9[7665]: #011at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default.patchOnByteBuddyAgent(AgentBuilder.java:9429) May 14 13:56:43 tomcat tomcat9[7665]: #011... 8 more ``` I've added my tomcat9 log from syslog for further help - [https://gist.github.com/Anandababu/a51571ce94e310c78abf65001ac75597](https://gist.github.com/Anandababu/a51571ce94e310c78abf65001ac75597) ---
**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\ **Post date:** [May 15, 2020, 12:37pm UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645/5 "2020-05-15T12:37:28Z")
Thanks for this comprehensive report @Ananda_Babu, I've managed to reproduce it locally and opened an issue to fix it [https://github.com/elastic/apm-agent-java/issues/1186](https://github.com/elastic/apm-agent-java/issues/1186). It seems the issue does not happen when only the `recording` property is switched from false to true (and it's even available in remote config). Thus in the mean time you should be able to use only the `recording` property and remove `instrument=false` from your properties to use the default value. The only difference with what you tried to do and using `instrument=true` (default) + `recording=false` is that it will use a no-op instrumentation and nothing is recorded. Doing like this also avoids any performance hit when enabling instrumentation afterwards as bytecode transformation is a short but cpu-intensive task. ---
**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\ **Post date:** [May 15, 2020, 2:31pm UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645/6 "2020-05-15T14:31:32Z")
Also, I've found that this issue does not happen with latest version of the agent released yesterday (1.16.0), and it behaves as you would expect. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 5, 2020, 10:31am UTC](https://discuss.elastic.co/t/unable-to-dynamically-update-instrumentation-true/232645/7 "2020-06-05T10:31:40Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.