# Missing transactions in the traceview after a call to DAPR

**URL:** <https://discuss.elastic.co/t/missing-transactions-in-the-traceview-after-a-call-to-dapr/329578>\
**Category:** APM\
**Tags:** dotnet, nodejs\
**Created:** [April 7, 2023, 10:18am UTC](https://discuss.elastic.co/t/missing-transactions-in-the-traceview-after-a-call-to-dapr/329578 "2023-04-07T10:18:01Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Khoroshko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_khoroshko/32/119542_2.png) [@Daniel\_Khoroshko](https://discuss.elastic.co/u/Daniel_Khoroshko)\
**Post date:** [April 7, 2023, 10:18am UTC](https://discuss.elastic.co/t/missing-transactions-in-the-traceview-after-a-call-to-dapr/329578/1 "2023-04-07T10:18:01Z")

</div>

As you can see the trace consists of many transactions:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/d/7/d7422026dfb6fc0449bd147b6bfe45c99a76aff4.png)

However when I open the first transaction I can see the trace chart not showing all the transactions, but just one of them

 ![MicrosoftTeams-image (2)](https://us1.discourse-cdn.com/elastic/original/3X/0/2/029c63f8fdeea668a297f52ebb88773f42f7f5a6.png)

What could be the reason for such behaviour? Could it be because of the wrong "parent.id" ?

The call to "localhost" is a service invocation call via DAPR. DAPR is a side cart container that listens to calls on localhost and forwards them to other DAPR containers belonging to other services in the network based on the value of the "dapr-app-id" header.

---

<div class="post-metadata">

**Author:** ![GregKalapos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregkalapos/32/37205_2.png) [@GregKalapos](https://discuss.elastic.co/u/GregKalapos)\
**Post date:** [April 8, 2023, 9:58am UTC](https://discuss.elastic.co/t/missing-transactions-in-the-traceview-after-a-call-to-dapr/329578/2 "2023-04-08T09:58:45Z")

</div>

Hi @Daniel_Khoroshko

> What could be the reason for such behaviour? Could it be because of the wrong "parent.id" ?

Yes, it could be, but I don't see enough info here to verify.

One important thing to note is that the `parent.id` in case of an outgoing call points to a span (and not a transaction!) in the upstream service. [Here is an example in a demo app](https://demo.elastic.co/app/apm/services/frontend-rum/transactions/view?kuery=&rangeFrom=now-15m&rangeTo=now&environment=ENVIRONMENT_ALL&serviceGroup=&comparisonEnabled=true&traceId=0c9e466c5e7f8f3c3671802a1f8b0085&transactionId=60a8e08fe9cc1d6e&transactionName=%2Fplace-order&transactionType=user-interaction&offset=1d&latencyAggregationType=avg&flyoutDetailTab=&waterfallItemId=) about this. You can check the `span.id` in case of an outgoing HTTP call and then check the `parent.id` of the transaction in the downstream transaction. Those are the ones that need to match (and not the `transaction.id` of the upstream service and the `parent.id` in the downstream service).

> The call to "localhost" is a service invocation call via DAPR. DAPR is a side cart container that listens to calls on localhost and forwards them to other DAPR containers belonging to other services in the network based on the value of the "dapr-app-id" header.

So, following the logic from above: you show a span named `GET localhost` on your 2. screenshot. I assume this is the very first service and it forwards the call to another DAPR container. This span has a `span.id` and then you have a 2. service where the `trace.id`s match across the 2 transactions. Could you please check if `parent.id` of the downstream transaction of the one you show on this screenshot matches the `span.id` of `GET localhost`? If that's not the case, then this is where it may go wrong.

---

<div class="post-metadata">

**Author:** ![Daniel\_Khoroshko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_khoroshko/32/119542_2.png) [@Daniel\_Khoroshko](https://discuss.elastic.co/u/Daniel_Khoroshko)\
**Post date:** [April 9, 2023, 11:12am UTC](https://discuss.elastic.co/t/missing-transactions-in-the-traceview-after-a-call-to-dapr/329578/3 "2023-04-09T11:12:39Z")

</div>

Hi Greg

Thank you very much for such a detailed response. I didn't know that the `span.id` was so important, and you are correct in saying that the downstream transaction has a wrong `parent.id` that doesn't point to the original span

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/7/07302072ba3c1c8471483d8ab5e8b5589c84f68a.png)

I've checked the properties of the downstream transaction and I can confirm that the second part of the traceparent header gets changed in DAPR. The good part is that the original header is preserved in the `Elastic-Apm-Traceparent` header.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/4/743cdfbcb36e2291c458bd4157c4cf705224e846.png)

I know that it is possible to export Open Telementry stats from DAPR and then import those into Elastic APM. However, this is not supported by the DAPR configuration that is exposed as a part of Azure Container Apps config.

I am going to investigate if it possible to export OTEL data to Elastic APM inside the azure-hosted DAPR and in case this is not possible I'll try to replace the `traceparent` header with the original one in the incoming requests, which should be good enough for our case

Thanks!

---

<div class="post-metadata">

**Author:** ![Daniel\_Khoroshko](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/daniel_khoroshko/32/119542_2.png) [@Daniel\_Khoroshko](https://discuss.elastic.co/u/Daniel_Khoroshko)\
**Post date:** [April 10, 2023, 8:59pm UTC](https://discuss.elastic.co/t/missing-transactions-in-the-traceview-after-a-call-to-dapr/329578/4 "2023-04-10T20:59:12Z")

</div>

I was able to solve the problem in nodejs services by prioritising elastic apm traceparent over a regular traceparent header by patching the http-shared.js file in the postinstall script. The file is located inside `node_modules/elastic-apm-node`. Not the most elegant solution, but gets the job done

Not sure how to deal with dotnet services in this case, will think of a similar solution

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 1, 2023, 4:59pm UTC](https://discuss.elastic.co/t/missing-transactions-in-the-traceview-after-a-call-to-dapr/329578/5 "2023-05-01T16:59:32Z")

</div>

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.
