We've recently done an APM Java agent upgrade from 1.17 to 1.30.1. For most of our APM monitoring things have gone smoothly but one of our services, all of the transaction names are coming through as hex values instead of the method names. Can anyone help with why this may have happened, and only on one service?
Hi @ethranes ,
Do you know what type of transaction is being captured here ?
What are the names that were previously used in those transactions ?
Also, could you share a copy of the transaction document ?
This is probably because we've added auto-instrumentation support for TimerTask in 1.18.0. Does BaseProcessor have TimerTask as a super class?
The hash might be a sign that the task is implemented as an anonymous inner class or as a lambda function.
Transactions that are created from a timer task have the type scheduled. The UI doesn't mix them with the request type. So if you can't see the HTTP request transaction of your application, you probably just need to switch the transaction type in the dropdown.
We were using 1.17 earlier and the type of the transaction was custom. Previously, the names of the transactions were like ABCProcessor, XYZProcessor where the hierarchy is like ABCProcessor extends BaseProcessor extends TimerTask. The names like ABCProcessor have been configured under trace_methods property in the elasticapm.properties file. The hexvalue#run (hex seems to be cglib proxy instance name) do show up as transaction type scheduled, but the there's no transactions with the name we've configured under trace_methods after the upgrade. Also, the hex values obviously change after the restart, so can't really identify transactions like that.
Thanks for the hint that it's a cglib proxy. I've filed an issue to prevent the TimerTask instrumentation from instrumenting proxies:
You have two options to work around the issue:
- Set
classes_excluded_from_instrumentationto*$Proxy*,*$$*,*$JaxbAccessor*,*CGLIB*,*EnhancerBy*. You can also remove thetrace_methodssetting as the transactions will be created by the dedicatedTimeTaskinstrumentation. - Set
disable_instrumentationstotimer-taskso that only thetrace_methodssetting will create transactions for your tasks.
2 Likes
@Swapnil_Sonawane please try this snapshot without your original trace_methods configuration (and without classes_excluded_from_instrumentation or disable_instrumentations configs if you added such) and let us know if you get properly-named transactions.
Make sure to check the scheduled transaction types for those.
1 Like
Thanks for your help both, we've opted to go for adding timer-task to disable_instrumentations for now. That has done the trick, but like you say, they're now registering as custom transactions, which is fine to be honest. We'll keep an eye on `TimerTaskInstrumentation` instruments proxies · Issue #2618 · elastic/apm-agent-java · GitHub and look to updating our agents again in a later build. Cheers!
There is already a fix for this issue - Exclude proxies in TimerTaskInstrumentation by eyalkoren · Pull Request #2620 · elastic/apm-agent-java · GitHub. What I asked is that you verify that this fix works as expected, so that you and other users will get everything traced as expected out-of-the-box.
Using the builtin transactions will also provide additional data for your transactions.
Whenever possible, it's best to rely on inherent agent capabilities, so doing this extra effort to test the fix will not only serve other users, but you as well.
Thank you.
Version 1.31.0 has been released with the related fix.
With this version, TimerTask transactions should be named properly without any special configuration.
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.