Elastic APM java not showings spans

Elastic Observability APM
1

Hi i have created a spring boot 3 application. Doing attach:
public static void main(String args) {
ElasticApmAttacher.attach();
SpringApplication.run(Application.class, args);
}

In apm only showing transaction DispatcherServlet#doGet
should show the correct mappings here? and about spans i get this message in the debug log

DEBUG co.elastic.apm.agent.pluginapi.CaptureSpanInstrumentation - Not creating span for 123#readFile because there is no currently active span.

service_name=pt2
application_packages=se.af.prestanda.spring
secret_token=xx
environment=xx
server_url=xx
verify_server_cert=false
log_level=DEBUG
recording=true
enabled=true
disable_instrumentations=servlet-api, servlet-api-async, servlet-api-dispatch, servlet-input-stream,
\ spring-mvc, spring-resttemplate
enable_experimental_instrumentations=true
capture_body=all
capture_headers=true

2

so no spans are showing up...

3

Everything works fine if i do this manually in the code:

    Transaction transaction = ElasticApm.startTransaction();
    transaction.setName("Controller#Metric");
    transaction.setType(Transaction.TYPE_REQUEST);

Span span1 = transaction.startSpan().setName("getWorkspaceId");
span1.end();

transaction.end();

4

Hi @johanostberg ,

What frameworks are you using in your application ? Are they listed in the supported technologies ? What types of spans do you expect to see ? Does your service calls a database and/or another service through HTTP ?

Here the DEBUG message that you reported seems to indicate the the agent is able to properly instrument the library/framework creating spans, but it is ignored because there is no active transaction (transaction is a top-level span that is required here).

Could you try by removing the disable_instrumentations configuration ? Does it makes a difference for the captured spans ?

If further analysis is required, please capture the agent logs using this procedure.

5

Hi im using springboot 3.0
in my Controller.java class:
@PostMapping("/postjson")
public String postBody(@RequestBody String jsonString) throws JSONException, IOException {
i expect this to show up as a transaction controller#postjson
and this call to show up as a span readFile(FileName);
no differens when i remove disable_instrumentations..

6

20T16:44:24.047+01:00 INFO 25396 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-12-20T16:44:24.047+01:00 INFO 25396 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2022-12-20T16:44:24.048+01:00 INFO 25396 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 1 ms
2022-12-20 16:44:24,090 [http-nio-8080-exec-1] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation CaptureSpanInstrumentation: ((false or name(startsWith(se.af.prestanda.spring))) and not((name(contains($Proxy)) or name(contains($$)) or name(contains($JaxbAccessor)) or name(contains(CGLIB)) or name(contains(EnhancerBy)))) and declaresMethods(whereOne(declaresAnnotations(whereOne(ofAnnotationType(name(equals(co.elastic.apm.api.CaptureSpan)))))))) matches class se.af.prestanda.spring.NeoloadCompare
2022-12-20 16:44:24,094 [http-nio-8080-exec-1] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation CaptureSpanInstrumentation: (declaresAnnotations(whereOne(ofAnnotationType(name(equals(co.elastic.apm.api.CaptureSpan))))) and not(isAbstract())) matches public static java.util.Map se.af.prestanda.spring.NeoloadCompare.readFile(java.lang.String) throws java.io.IOException
2022-12-20 16:44:24,120 [http-nio-8080-exec-1] DEBUG co.elastic.apm.agent.bci.IndyPluginClassLoaderFactory - Creating plugin class loader for jdk.internal.loader.ClassLoaders$AppClassLoader@1d44bcfa containing [co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$SetNameInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.CaptureSpanInstrumentation, co.elastic.apm.agent.pluginapi.TracedInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$IsSampledInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$EndWithTimestampInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ApiScopeInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$GetTraceIdInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetDestinationServiceInstrumentation, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CurrentSpanInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$EnsureParentIdInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CurrentTransactionInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$EndInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$IsSampledInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$GetIdInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetResultInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CaptureExceptionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddNumberLabelInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$LegacyCaptureExceptionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetStartTimestampInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$CaptureExceptionInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetDestinationAddressInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$SetTypeInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$ActivateInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$IsSampledInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$SetNameInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetNameInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$SetTypeInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$GetIdInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$InjectTraceHeadersInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetStartTimestampInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$InitializeInstrumentation, co.elastic.apm.agent.pluginapi.CaptureSpanInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddNumberLabelInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddStringLabelInstrumentation, co.elastic.apm.agent.pluginapi.HeaderInjectorBridge, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetFrameworkNameInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$GetIdInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$ActivateInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$AddTagInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.CaptureTransactionInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetTypeInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetDestinationAddressInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$GetTraceIdInstrumentation, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CurrentTransactionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.package-info, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$InjectTraceHeadersInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$StartTransactionInstrumentation, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetResultInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$ActivateInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$DoCreateSpanInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$IsSampledInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.CaptureExceptionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.HeaderExtractorBridge, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CurrentSpanInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$EndInstrumentation, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$StartTransactionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetTypesInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$EndInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$AddCustomContextInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$DoCreateSpanInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$StartTransactionWithRemoteParentInstrumentation, co.elastic.apm.agent.pluginapi.TransactionInstrumentation, co.elastic.apm.agent.pluginapi.Utils, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$GetTraceIdInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$DoCreateSpanInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CaptureExceptionInstrumentation, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$AddCustomContextInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.CaptureTransactionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$CaptureExceptionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetTypeInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$CaptureExceptionInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetDestinationServiceInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$DoCreateSpanInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddBooleanLabelInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$AddTagInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$CaptureExceptionInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$DoCreateExitSpanInstrumentation, co.elastic.apm.agent.pluginapi.TracedInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetOutcomeInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.HeadersExtractorBridge, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetUserInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$EndInstrumentation, co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$StartTransactionWithRemoteParentInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$LegacyCaptureExceptionInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddBooleanLabelInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$EndWithTimestampInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddStringLabelInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetUserInstrumentation, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$EnsureParentIdInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetTypesInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$GetIdInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$DoCreateExitSpanInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.HeadersExtractorBridge$Extractor, co.elastic.apm.agent.pluginapi.ApiInstrumentation, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$ActivateInstrumentation, co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$GetTraceIdInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetOutcomeInstrumentation, co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetFrameworkNameInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$InitializeInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetNameInstrumentation$AdviceClass, co.elastic.apm.agent.pluginapi.ApiScopeInstrumentation, co.elastic.apm.agent.pluginapi.CaptureExceptionInstrumentation, co.elastic.apm.agent.bci.classloading.LookupExposer]
2022-12-20 16:44:24,135 [http-nio-8080-exec-1] DEBUG co.elastic.apm.agent.pluginapi.CaptureSpanInstrumentation - Not creating span for NeoloadCompare#readFile because there is no currently active span.

7

The agent debug logs are not required for now (I was just anticipating a bit here).
Have you tried removing disable_instrumentations configuration ?

When you created manually a transaction by using the agent Public API, the spans were properly captured, thus it confirms that only the transaction is missing.

You seem to be using Spring MVC framework, but you disabled its instrumentation in disable_instrumentations also servlet instrumentation (which is used to create transactions in this case) is also disabled which is very likely the cause of the issue here.

8

Thank you! I forgot to remove the disable_intrumentation and it now works kinda.
every transaction is named DispatcherServlet#doGet it should pickup the reqeustmapping that im calling right? And for the methods that are just doing stuff internally it shows upp like a span.
But i have another method that is doing a api call to another application for that one i just get the api call (GET web-api.app).

9

Ok i deployed my app in ocp and now it everything works thank you!!!

10

update now i only get do DispatcherServlet#doGet for all my transactions i dont understand why this is happening?

11

Did you completely removed the option or did you just removed servlets from disable_instrumentations ?

The reason I am asking this is because the Spring-MVC one is in charge of setting a proper name and would thus allow to better "qualify" the generic Servlet endpoint that is common to all of them.

12

i removed the whole option.

13

In this case, could you provide a code sample that shows how your endpoints are defined ?
In some cases setting use_path_as_transaction_name might be useful : HTTP configuration options | APM Java Agent Reference [1.x] | Elastic

However, that can be an issue if you are using parts of the URL as parameters, which is common with REST endpoints, but at least you could see if that makes a difference.

14

with use_path_as_transaction_name i get different transaction names for my endpoints but i dont see the span anymore:
@RequestMapping(value = "metrics")
public ResponseEntity metrics() throws IOException {

        return ResponseEntity.ok()
                .headers(headers)
                .body(readFile(c:/temp/1.txt);

}

15

Should spans on internal metods be created by default or do i need to annotate @CaptureSpan for every method?

16

Hi,

From the annotations it seems that you are using Spring MVC, so this should definitely be supported, could you provide the version of the Spring dependencies that you are using ?

Changing the use_path_as_transaction_name value does not have any impact on spans capture, it only impacts how the transaction is named.

Spans are automatically captured for any instrumented method, so it means mostly external services calls like through the JDBC API (or any of the supported technologies).

If you want to also create spans for methods that are not instrumented automatically, you can either instrument them with trace_methods or annotate them like you suggested.
Using @CaptureSpan will only create spans if the method is called within a transaction, it won't create a transaction.

17

thank you everything works perfect now. Just want to check can we do something similar with .net?

18

Yes, this should also be doable with dotnet if needed. However I don´t know the details, thus check documentation and open other posts in forum if you need further help.

19

ok thank you

20

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.