Intrument some classes or methods instead of profiling sampling

Elastic Observability APM
1

Java agent supports many technologies instrumentations , but not all.

In the latest release version 1.15.0, creating spans for any method executions can be enabled, but the sampling is inaccurate because the interval of sampling can not be setting too low.

In addition, the profiling only supports the configuration by class level, and the method level will be better.

so how about intrument the methods which are configured, and creating spans for their executions, instead of profiling sampling ?

2

You can certainly do that via the trace_methods configuration option. But be sure to read the caveats as this options is generally not recommended to use for a large number of methods or for fast executing ones.

Out of curiosity, why is the accuracy of the sampling profiler not enough for you? Sure, it's only accurate up to profiling_inferred_spans_sampling_interval but shouldn't that be enough to identify slow methods?

3

Sampling profiler can identify slow methods, but it can not show how slow it is. If intrumenting methods is supported, it would be the better choise.

the trace_methods configuration option is ok, thank you!

4

Both have their tradeoffs but for production usage I'd strongly recommend the sampling profiler over trace_methods.

5

Why? The trace_methods is archieved through the instrumentation of some methods while application startup? it might be cheaper , i think.

6

For trace_methods, there is an overhead that is associated with each instrumented method. Let's assume the overhead is 1µs. If you instrument methods that are executed millions of times within a single request and normally just take 1ns, you'll see massive overhead.
While this may sound like an extreme example, it's not uncommon that methods are executed in a tight loop. We've got several reports from users that their performance degraded as a result of overusing trace_methods. While it's a great tool it's also easy to shoot yourself in the foot with it.

See also this blog post: https://www.elastic.co/blog/from-distributed-tracing-to-distributed-profiling-with-elastic-apm

7

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.