Opentelemetry traces in event driven system

Elastic Observability APM
1

Elasticsearch observability: 8.2

Is there anything special in your setup?
Normal POC setup:

  • two services
  • HTTP and message bus communication using Dapr
  • Opentelemetry collector for log and trace aggregation
  • Using otlp/elastic exporter

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
I'm trying to collect traces and logs using cloud solutions for observability. I'm collecting logs and traces with OpenTelemetry collector. For service-to-service HTTP communication traces have a nice timeline with spans across all services. Also, logs are aggregated as they should be.

When I try to use event-driven approach using Dapr traces are still collected, but the spans are not displayed as they should be. I only see span from the originating service in the timeline. Although logs are there from originating service and the service subscribed to the log. Meaning that trace id is ok both in publisher and subscriber. I tested the same with Newrelic and there all spans are present in the request timeline.

Did anyone manage to set up APM with event-driven microservices?
Steps to reproduce:

  1. Call service A
  2. Service A publishes a message to a topic
  3. Service B is subscribed to that message and processes it.

Errors in browser console (if relevant): No errors

2

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.