Distributed Tracing

Are there plans to support distributed traces in APM? (Or perhaps it already does?) I'd like to have a set of spans created across services that are then linked together on the APM server. From reading the OpenTracing specification, this seems to be how they imply a tracing implementation would work because spans are only ever linked by passing information "forward". Concrete example:

  • Service Foo starts a span A with some number of child spans representing different execution segments within Foo.
  • Foo then calls (RPC/HTTP/message bus/whatever) service Bar, passing forward some of the span context (at least the id)
  • Bar extracts the context from all requests, so it extracts the context from Foo and creates a span B referencing the span from the context (in OpenTracing, either as a child span or a follows from span).
  • Bar processes with some number of child spans for its execution.
  • Bar responds to Foo and ends span B.
  • Foo receives the response, creates some number of additional spans for further execution and then ends span A.

In the above sequence of events, Foo never explicitly connects span A back to the spans from Bar, in fact, from Foo's perspective it has no idea if Bar is actually going to create additional spans. Similarly, Bar has no idea if Foo is going to create additional spans after it responds. Would/does the APM API support such a scenario? From reading the documentation, it seems like I might need to respond with the spans from Bar back to Foo so that it can include those spans in the transaction payload.

It's not possible yet, but distributed tracing is something we are working towards. I can't say yet whether we'll implement OpenTracing, but it's certainly under consideration, along with some alternatives like OpenCensus.

2 Likes

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.