OpenTelemetry Plans for Elastic APM

[digitalron (Ronald Tumulak)]

I came across a discussion here: Incompatibility while using co.elastic.apm:apm-opentracing:1.6.1 and elastic-apm-agent-1.6.1 where OpenTelemetry was mentioned.

As this is an item I've been looking into (I also evaluated OpenTracing and OpenCensus before deciding to adopt Elastic as our stack), does Elastic APM have any plans to consider OpenTelemetry support? We are going live with our Elastic APM Observability stack sometime June/July this year and I'm preparing plans to expand its use in our organisation. As we are also fans of CNCF technologies we'd like to know if OTel is something we should plan for in the near future.

Thanks!

[felixbarny (Felix Barnsteiner)]

We are definitely exploring different ways how to integrate with OpenTelemetry.

The first level of integration will be to write a golang exporter. With that in place, you can use the go reference implementation (likely derived from https://github.com/census-instrumentation/opencensus-go) of OpenTelemetry to send the traces to APM Server. It also allows to use what's currently know as OpenCensus Service and OpenCensus Agent (see https://github.com/census-instrumentation/opencensus-service) to send traces to APM Server.

Another point of integration would be to implement bridges from the OpenTelemetry API to our agents so that it's possible to mix-and-match auto instrumentation provided by our agents with manual instrumentation using the OpenTelementy API.

Which kinds of integrations would you be most interested in? Do you feel the need of having to do a lot of manual instrumentation on top of the auto instrumentation our agents provide?

Cheers,
Felix

[digitalron (Ronald Tumulak)]

Thanks for the quick response Felix.

Our interest lies in the ability to use agent-based instrumentation as much as possible on technologies not immediately covered by Elastic. As an example, I am looking at SpecialAgent as a possible way to expand automated tracing for things like Rabbit, Kafka, and Cassandra which we use.

Another area of interest to us is the ability to create trace propagation libraries for the likes of our legacy C++ applications - we feel there is minimal clamour to support this, but having a way to continue traces here would be super. Bridges are interesting to us here as we know not everything can be automatically instrumented. At the very least, trace ID propagation with some form of baggage support (to allow custom info to be sent across services as well) would be nice.

I guess the bottom line for us is having standard tracing and metrics interfaces/constructs that we can aim for so that we can have interchangeable agents and intermediate collectors that can all be pointed to Elastic as our final collector and Visualisation stack.