# Apm-agent not working, after updating war in tomcat **URL:** \ **Category:** APM\ **Tags:** java\ **Created:** [July 3, 2020, 8:09am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750 "2020-07-03T08:09:17Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![bob96589](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bob96589](https://discuss.elastic.co/u/bob96589)\ **Post date:** [July 3, 2020, 8:09am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/1 "2020-07-03T08:09:18Z")
**Kibana version** : 7.4.1 **Elasticsearch version** : 7.4.1 **APM Server version** : 7.6.1 **APM Agent language and version** : java, apm-agent-attach-1.17.0.jar **Browser version** : **Original install method (e.g. download page, yum, deb, from source, etc.) and version**: docker **Fresh install or upgraded from other version?** Fresh install **Is there anything special in your setup?** [https://www.elastic.co/guide/en/apm/agent/java/current/setup-javaagent.html#setup-tomcat](https://www.elastic.co/guide/en/apm/agent/java/current/setup-javaagent.html#setup-tomcat) **Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**: I have tomcat with serveral java web services (serveral war files) on it. After setting up apm-agent in tomcat and apm-server, I can successfully send the data to kibana. It all works well so far. But when I need to get one of my war updated. Here are my steps： - remove one war file from webapps, the other wars remains the same - send a new war (with the same name as the one I deleted) to webapps With settings in server.xml ``` ``` I didn't restart my tomcat. The new war automatically get deployed and unpacked. And the updated service works well. But In kibana, **I can't receive data from the updated service anymore.** The other services still work fine. ---
**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\ **Post date:** [July 7, 2020, 6:49am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/2 "2020-07-07T06:49:26Z")
Hi @bob96589, welcome to the forum. Can you provide the DEBUG logs of the agent when doing the re-deploy operation ? (see [documentation](https://www.elastic.co/guide/en/apm/agent/java/current/trouble-shooting.html#trouble-shooting-logging)) What versions of the JDK and Tomcat are you using ? ---
**Author:** ![bob96589](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bob96589](https://discuss.elastic.co/u/bob96589)\ **Post date:** [July 8, 2020, 8:39am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/3 "2020-07-08T08:39:38Z")
Hi @Sylvain_Juge. Thanks for your reply. The following link is DEBUG logs of the agent, and it consists - apm-agent start - trigger service api (SpcAlphaParamController#getSpcAlpha) - apm-agent successfully send data to apm-server, and can see data in kibana - update war in tomcat - trigger service api again (SpcAlphaParamController#getSpcAlpha) - cannot see data in kibana > **docker images:** tomcat:9-jdk8 in the container java version is 1.8.0\_252 **APM Agent version:** apm-agent-attach-1.16.0.jar (Sorry, in the previous post, I wrote the wrong version) ---
**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\ **Post date:** [July 9, 2020, 8:26am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/4 "2020-07-09T08:26:04Z")
Hi @bob96589, I was unable to reproduce the behavior that you observed locally with the following configurations: - agent 1.17.0 + tomcat 8.5.54 + OpenJDK 8u242 - agent 1.16.0 + tomcat 9.0.37 + OpenJDK 8u242 In agent debug logs: - there is no error while sending data to `apm-server` - sample rate is at default value, thus all transactions should be captured - I can't really pinpoint the time where the re-deploy happens - the transaction seems to be still created after re-deploy (and we can see some SQL queries spans), but there is only a few of them. Thus I have few other questions: - is this issue limited to transactions ? for example, are other metrics (like JVM metrics) still sent to Kibana after re-deploy ? - did you tried with multiple requests after the re-deploy ? (sounds really obvious but I'd better be safe). For example, if there was previously 100 transactions/s, do you see none of them after re-deploy even waiting for a few minutes ? - does restarting the whole JVM makes it work again ? ---
**Author:** ![bob96589](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bob96589](https://discuss.elastic.co/u/bob96589)\ **Post date:** [July 13, 2020, 3:39am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/5 "2020-07-13T03:39:07Z")
Hi @Sylvain_Juge, Answer the question: - Is JVM metrics stored in apm-7.6.1-metric-\* index? In apm-7.6.1-metric-\* index, I can only find data with service.name equals to "tomcat-application". And even after re-deployed, the data is consistently being sent to the index. - Yes. I did try multiple requests. But the log in the previous post, I only did a successful and a fail request to avoid too much log. - Yes. Restarting the JVM (Restarting tomcat) fixs the problem. Following link is a animated gif that demonstrates how I reproduce the problem. The configuration is agent 1.16.0 + tomcat 9.0.37 + OpenJDK 8u242. > [https://github.com/bob96589/temp/blob/master/error.gif](https://github.com/bob96589/temp/blob/master/error.gif) ---
**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\ **Post date:** [July 26, 2020, 7:10am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/6 "2020-07-26T07:10:01Z")
The end of the log you uploaded clearly shows the agent still traces `SpcAlphaParamController#getSpcAlpha` after redeployment and it even seems as it sends these data. So, my suspicion is that there is a problem related to the service name of the redeployed app. Please try the followings and provide feedback: 1. Try setting agent's `log_level` to `TRACE`. This will print a lot of info, part of which is the full trace documents sent to the APM server. The [communication with APM server](https://www.elastic.co/guide/en/apm/server/current/events-api.html) relies on NDJSON, so please look for metadata JSONs sent before and after redeployment and upload them. 2. It seems that the agent is using the auto-discovered service name. Please try setting the [`service_name`](https://www.elastic.co/guide/en/apm/agent/java/current/config-core.html#config-service-name) config and see if the problem is resolved. ---
**Author:** ![bob96589](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bob96589](https://discuss.elastic.co/u/bob96589)\ **Post date:** [August 4, 2020, 7:42am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/7 "2020-08-04T07:42:47Z")
Hi @Eyal_Koren, 1. Here is the log with log\_level TRACE. entire log: [https://gist.github.com/bob96589/1b478f03785375fa04f1d9f4c4e7ebde](https://gist.github.com/bob96589/1b478f03785375fa04f1d9f4c4e7ebde) transaction log sent before deploy (extract from entire log): [https://gist.github.com/bob96589/8165c00e70686e571378a2b06bdc8889](https://gist.github.com/bob96589/8165c00e70686e571378a2b06bdc8889) transaction log sent after deploy (extract from entire log): [https://gist.github.com/bob96589/82f9a5c40068b5302ee4c23d2cd5a6e2](https://gist.github.com/bob96589/82f9a5c40068b5302ee4c23d2cd5a6e2) I noticed a difference between the tranction logs being sent before and after redeploy. **Before redeploy** , the json being sent to kibana consists service.name, "SPC-Service-1-0", which is an auto-dicovered service name. After data being sent to kibana, I can see the data in "SPC-Service-1-0" service. **After redeploy** , service.name in the json was missing. **(Probably it is a bug?!)** After data being sent to kibana, I can see the data in "tomcat-application" service. So No matter redeploy or not, the data is always being sent and can be received in kibana, but categorized in different service name. 1. Setting the service\_name config solves the problem and makes the data always stay in the right service name. ---
**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\ **Post date:** [August 4, 2020, 3:13pm UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/8 "2020-08-04T15:13:25Z")
Extremely useful info, thanks! I think I know what the problem is, will try a fix tomorrow ---
**Author:** ![Eyal\_Koren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eyal_koren/32/36830_2.png) [@Eyal\_Koren](https://discuss.elastic.co/u/Eyal_Koren)\ **Post date:** [August 5, 2020, 7:15am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/9 "2020-08-05T07:15:30Z")
@bob96589 Please try [this fix](https://apm-ci.elastic.co/blue/organizations/jenkins/apm-agent-java%2Fapm-agent-java-mbp/detail/PR-1326/1/artifacts) and let me know if it resolves the problem ---
**Author:** ![bob96589](https://avatars.discourse-cdn.com/v4/letter/b/d26b3c/32.png) [@bob96589](https://discuss.elastic.co/u/bob96589)\ **Post date:** [August 7, 2020, 7:10am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/10 "2020-08-07T07:10:46Z")
Hi @Eyal_Koren, Problem solved. Thanks. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 28, 2020, 3:10am UTC](https://discuss.elastic.co/t/apm-agent-not-working-after-updating-war-in-tomcat/239750/11 "2020-08-28T03:10:53Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.