Hello Team,
I need your support with distributed tracing implementation for my self hosted services running using .NET8 and Elastic.APM.AspNetCore version 1.30.0.
There are 3 self hosted services and a client (developed with .NET8) and services are communicating with each other thouugh kafka message communication.
There a client initiating request by generating distributed trace id and passing it to the next service and same is passed till the last service.
Request and Response workflow:
Client -> Service1 (Request) -> Service2 (Request) -> Service3
Client <- Service1 (Response) <- Service2 (Response) <- Service3
Example code for generating distributedTraceId in client:
Class Client
{
public void Run()
{
var transaction = Agent.Tracer.StartTransaction($"TransactionName", "TransactionType");
var distributedTraceId = (Agent.Tracer.CurrentSpan?.OutgoingDistributedTracingData ?? Agent.Tracer.CurrentTransaction?.OutgoingDistributedTracingData)?.SerializeToString();
// Some Code Here
var requests = new List<ServiceRequest> { new ServiceRequest() };
var response = client.SendRequest(requests.ToObservable(), true).ToList().Wait();
transaction.End().
}
}
When client sends request to service then service is processing request and sending response after processing request and calling other services in pipeline.
Each service is fetching received distributedTraceId from caller and passing it to next service.
Example service code for fetching passed distributedTraceId and passing to next service:
Class Service1
{
private readonly ITracer _tracer;
public Service1(ITracer tracer)
{
_tracer = tracer;
}
public void Func1(ServiceRequest request)
{
var transaction = _tracer.StartTransaction($"Service1TransactionName", $"Service1TransactionType", DistributedTracingData.TryDeserializeFromString(request.DistributedTraceId));
// Some Code Here
var service2Request = new Service2Request();
service2Request.DistributedTraceId = request.DistributedTraceId
// This service2Request object is passed to service2
transaction.End();
}
}
In the end response is getting back the generated DistributedTraceId, but the trace data is not shown in the APM.
Could you please tell, is this implementation has any issue and how can I get the distributed traces in APM for this type of implementation?