Javaagent - explicitly defining a class to be visible in dependencies, transactions

Kibana version: 8.4.3

Elasticsearch version: 8.2.3

APM Server version: 8.2.3

APM Agent language and version: 1.33

Hi - we have a user who has added the elastic agent to a third party application. One known dependency (an ldap backend) isnt showing but is showing when appdynamics is deployed instead of elastic agent.

Q1 - is there a way to explicitly define a class or classes to be instrumented in the elastic agent config ?
Q2 - is there any way to see what class files have been loaded as the ldap classes are part of the 3rd party application so the class names are not known

Hi @dave.richards ,

Currently the Java agent does not provide out-of-the-box instrumentation for LDAP services, as a result this is normal that you don't see it in the service dependencies.

In this case, assuming that you can't modify the application, you have a few options to capture the outgoing spans:

If you don't have access to the sources of the application, you can try the following techniques to know which classes/methods to instrument:

  • search in the application files for the dependencies, it is likely using an open-source library for LDAP implementation. If you have it, then it is mostly a matter of identifying how the library is used by the application.
  • start the application with a debugger, if you have the exact same version of the LDAP library in your IDE project you will be able to place breakpoints and identify how the application is using this library
  • use a sampling profiler like async-profiler and capture samples when the application calls LDAP (for example when a user connects), you should see the invoked methods in the captured samples.

Thanks for the quick reply. Coiuyld you share an example of a the trace_methods config setting assuming that the ldap classes were held in a package called com.ldap ?

Also - will the elastic agent be getting updated to pick up ldap backends, as the user is evaluating this against appd so in a direct comparison this is a weakness against the elastic java agent

Thanks

In this case trace_methods=com.ldap.* should instrument all the methods of all classes in the
com.ldap package, which is very probably not what you want as not everything is expected to be relevant and instrumenting everything implies an important overhead.

You can find other examples in the documentation with finer matching using method name and argument types.

Regarding adding official support for LDAP, there is no standard API, so adding it to the agent means that the implementation will be specific to this LDAP client implementation. Do you know what library is being used here ?
We haven't had much similar requests so far, so I doubt that we will add it in a short/medium time frame.

However, there is still the option to create an external plugin for it, if you have the name and version of the LDAP client library that would be a good start.

Another alternative would be to try with OpenTelemetry instrumentation agent, the APM server supports native OTLP intake so it could be used as an alternative Java agent.

OK noted, thanks for your help and advice. Im going to try and use the trace_methods option, we have a call with our user today so will see how that goes. Noted also about not using com.ldap.*, that was just for me to see the syntax. We will try and identify the correct candidate classes with the user and pin it down to those. Thank you.

Hi - setting trace_methods to both
trace_methods=com.sun.jndi.ldap*
and
trace_methods=ldap
did not give us the desired results. From the 2nd config setting we can see some ldap references now in the discover view and also in the transactions, but we are still not seeing the ldap backend as a dependency or on the service map.

Ive raised a case with elastic for this now, and in the meantime we will take a look at open telemetry using the Otel java agent. Do you have any example otel properties files we could look at, specifically around the area of exposing ldap backends ?

Thanks

Hi,

Yes, using trace_methods does not allow to capture the backend as it only allows to capture a transaction/span. The ability to capture the service dependency requires code that captures the relevant data and stores it into the agent/otel data structures.

There is nothing specific in OTel semantic conventions regarding ldap attributes, thus I guess that trying to capture the generic span attributes like host and port should be the best option for now. I haven't tested but it is likely that the apm server will automatically create destination from those generic attributes.

The LDAP client you are mentioning, is that what is included in the JDK itself JNDI/LDAP Service Provider ? Have you checked the application dependencies (.jar files) to be sure there isn't a higher-level client used ?

When you say "raised a case with elastic", does it means you have also reached out to elastic support or does it only refers to this conversation on our public forum ? The reason I am asking this is because the forum is only "best effort" so you should not have very strong expectations for timely replies here, whereas a support case does.

Hi, I have raised a support case with elastic for this issue. Ill be reaching out to the user today, but this is a 3rd party application and we dont have access to the source code so not 100% sure yet what LDAP service they are using. Thanks

Hi - one other question. We are looking at the open telemetry java agent to see if the ldap backend is detected with that. Is there an option to disable SSL in the otel agent, similar to this setting for the elastic agent

-Delastic.apm.verify_server_cert=false

maybe OTEL_EXPORTER_OTLP_INSECURE, I'm not sure if the OTEL java agent supports it

Thanks will try that

Hi - we have got the otel java agent deployed but the same issue persists with that agent, still not detecting calls to LDAP (Microsoft AD implementation). Thanks for all the suggestions, we will work with Elastic support going forward.

Hi @dave.richards ,

An update here: We are currently working on out-of-the-box LDAP support for the agent.
I have created a snapshot build which should automatically capture LDAP spans correctly and also make them visible in the service-graph / service dependencies:

Link to snapshot JAR

Would it be possible for you to try this snapshot agent and give some feedback whether it captures all the data you would like to see?

Hi Jonas, yes Il will download that and make it available to the team for testing.