# ELK APM Transaction not showing end to end flow if we have two diffent services **URL:** \ **Category:** APM\ **Tags:** dotnet, java\ **Created:** [May 19, 2020, 4:39am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242 "2020-05-19T04:39:46Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![Vissu\_Reddy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vissu_reddy/32/68599_2.png) [@Vissu\_Reddy](https://discuss.elastic.co/u/Vissu_Reddy)\ **Post date:** [May 19, 2020, 4:39am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/1 "2020-05-19T04:39:46Z")
We have two services called **ServiceA** and **ServiceB**. **ServiceA** has been implemented using Java language and **ServiceB** has been implemented using .Net language. Both these services has been integrated with respective ELK APM agents. Also we instrumented code both these services using Agents and were able to see ELK APM metrics separately in APM UI in Kibana but not in Single transaction view Here is Question. We are calling **ServiceB** from **ServiceA** , We were able to see ELK APM metrics separately in APM UI in Kibana not in Single transaction View. For **ServiceA** apm metrics are showing under **ServiceA** and for **ServiceB** apm metrics are showing under **ServiceB** in ELK APM UI. It’s not showing end to end flow in single transaction. It’s very hard to get mapping from **ServiceA** to **ServiceB** since both of these metrics are not in single transaction. If I want to get mapping between **ServiceA** and **ServiceB,** manually I have to verify for **ServiceA** and **ServiceB** apm metrics Is there is way to display **ServiceA** and **ServiceB** apm metrics in single view Transaction? Is there is way to display end to end flow in single view Transaction? ---
**Author:** ![nugusbayevkk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nugusbayevkk/32/126683_2.png) [@nugusbayevkk](https://discuss.elastic.co/u/nugusbayevkk)\ **Post date:** [May 19, 2020, 4:58am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/2 "2020-05-19T04:58:46Z")
Hi, What do you mean by "metrics" ? Can you share screenshot? ---
**Author:** ![Vissu\_Reddy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vissu_reddy/32/68599_2.png) [@Vissu\_Reddy](https://discuss.elastic.co/u/Vissu_Reddy)\ **Post date:** [May 19, 2020, 5:11am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/3 "2020-05-19T05:11:41Z")
I mean Transaction.. while we are calling from ServiceB from ServiceA, there is one transaction has been created for ServiceA and another transaction has been created for ServiceB. we were able to see two diffent transactions has been created in ELK APM UI. is there is way to put these two transactions into one transaction so that we can easy to map it down ---
**Author:** ![nugusbayevkk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nugusbayevkk/32/126683_2.png) [@nugusbayevkk](https://discuss.elastic.co/u/nugusbayevkk)\ **Post date:** [May 19, 2020, 5:23am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/4 "2020-05-19T05:23:35Z")
It's about [distribute tracing](https://www.elastic.co/blog/distributed-tracing-opentracing-and-elastic-apm), if you use supported technologies, you should see all information in transaction details of ServiceB. Can you share screenshot from ServiceB transaction details? ---
**Author:** ![Vissu\_Reddy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vissu_reddy/32/68599_2.png) [@Vissu\_Reddy](https://discuss.elastic.co/u/Vissu_Reddy)\ **Post date:** [May 19, 2020, 5:37am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/5 "2020-05-19T05:37:44Z")
I can’t share screen shot as its company proprietary As I said, ServiceA implemented using Java language and integrated with ELK Java APM agent and ServiceB implemented using .Net language and integrated with ELK .Net APM agent If we see distributed tracing there were two separated transactions are created one for ServiceA and another for ServiceB In ServiceA transaction details I don’t see ServiceB information details I just all about to know is there is way to combine these two transactions into one using distributed tracing? ---
**Author:** ![nugusbayevkk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nugusbayevkk/32/126683_2.png) [@nugusbayevkk](https://discuss.elastic.co/u/nugusbayevkk)\ **Post date:** [May 19, 2020, 5:41am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/6 "2020-05-19T05:41:49Z")
Ok, thanks. How does serviceA communicate with serviceB? What technology do you use? ---
**Author:** ![Vissu\_Reddy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vissu_reddy/32/68599_2.png) [@Vissu\_Reddy](https://discuss.elastic.co/u/Vissu_Reddy)\ **Post date:** [May 19, 2020, 6:31am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/7 "2020-05-19T06:31:26Z")
Its just a another api ... We are calling serviceB api in ServiceA. ServiceA - Java ServiceB - .Net can we connect thru bridge is it posible ? so that I'll show you my usecase easy for undestand. ---
**Author:** ![erjohnson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erjohnson/32/68847_2.png) [@erjohnson](https://discuss.elastic.co/u/erjohnson)\ **Post date:** [May 21, 2020, 9:33pm UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/8 "2020-05-21T21:33:31Z")
I have this issue as well, though both of my applications are .NET 4.8. I set up out-of-the-box MVC ("Web") and Web API ("API") applications, added the Elastic.Apm.AspNetFullFramework 1.5.1 NuGet package, and configured them to talk to our APM Server. Both APM Server and Kibana are 7.6.1. I then added a call to the API from the MVC app. They are both recording their individual transactions correctly, and the transactions are linked in the index by a shared trace ID, but Kibana only shows the Web transaction for the trace: ![trace1](https://us1.discourse-cdn.com/elastic/original/3X/5/2/526418b8233b53eda678aac7f857909e31207cd8.png) ![trace2](https://us1.discourse-cdn.com/elastic/original/3X/9/c/9c60e58ae17f97f4200b523482c2d2f7d72ec45d.png) ---
**Author:** ![erjohnson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erjohnson/32/68847_2.png) [@erjohnson](https://discuss.elastic.co/u/erjohnson)\ **Post date:** [May 21, 2020, 10:26pm UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/9 "2020-05-21T22:26:22Z")
The exact same setup, except using .NET Core apps, works fine: ![trace3](https://us1.discourse-cdn.com/elastic/original/3X/b/7/b715412cd2510b0dae5e071f3864e76a6aa41b0b.png) Is there some other metadata, beyond the trace ID, necessary to create the distributed trace waterfall which the FullFramework client library isn't populating? ---
**Author:** ![erjohnson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erjohnson/32/68847_2.png) [@erjohnson](https://discuss.elastic.co/u/erjohnson)\ **Post date:** [May 21, 2020, 11:03pm UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/10 "2020-05-21T23:03:03Z")
After some more digging, this appears to either be an issue with `HttpDiagnosticsSubscriber` or the way I'm using it. Notice that in the Core screenshot, there is an intermediate span for the HTTP request from the MVC app to the API app, while the only two spans associated with the trace in the Full Framework screenshot are the respective transactions. Indeed, the parent ID of the API transaction is not the Web transaction: ![trace4](https://us1.discourse-cdn.com/elastic/original/3X/0/1/016425a8b07d769d986903933e4177aa0e5ad476.png) Furthermore, there is no span with that ID: ![trace5](https://us1.discourse-cdn.com/elastic/original/3X/7/d/7d4249f242a83ebc5fce2d3cc43e69c611a7e940.png) As expected, in the Core setup the parent ID of the API transaction corresponds to the Web-\>API HTTP request span: ![trace6](https://us1.discourse-cdn.com/elastic/original/3X/a/7/a7d5e51a223aaba95590dd0f8a2c4d06d177994f.png) I had tried to add auto-instrumentation to `HttpClient` in the Full Framework apps with this code in Global.asax.cs: ```auto private static volatile bool apmInitialized; public override void Init() { if (!apmInitialized && Agent.IsConfigured) { apmInitialized = true; Agent.Subscribe(new HttpDiagnosticsSubscriber()); } } ``` Is that incorrect? I did verify that `Agent.Subscribe` is being called as expected, and before the outgoing HTTP request is sent. Even when commented out in the Web app, the API transaction is associated with a nonexistent parent ID. ---
**Author:** ![erjohnson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erjohnson/32/68847_2.png) [@erjohnson](https://discuss.elastic.co/u/erjohnson)\ **Post date:** [May 22, 2020, 11:16pm UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/11 "2020-05-22T23:16:35Z")
After spending a lot of time with the AspNetFullFrameworkSampleApp sample code, I figured out the issue. First of all, `HttpDiagnosticsSubscriber` is enabled by default in the Full Framework package, in the HTTP module: ```auto private static bool InitOnceForAllInstancesUnderLock(string dbgInstanceName) => InitOnceHelper.IfNotInited?.Init(() => { SafeAgentSetup(dbgInstanceName); _isCaptureHeadersEnabled = Agent.Instance.ConfigurationReader.CaptureHeaders; Agent.Instance.Subscribe(new HttpDiagnosticsSubscriber()); }) ?? false; ``` I think this is different from how the Core packages & API behave, where you have to explicitly enable anything other the [ASP.NET](http://ASP.NET) instrumentation. Second, even though it's enabled by default, the spans it creates won't actually be instrumented and captured unless the Microsoft.ApplicationInsights.Web package & module are installed. Once that module is added, the HTTP client spans are captured as expected, and the API application transaction can be linked to the Web application transaction, creating a distributed trace as expected. I don't know if either of those things are documented anywhere, but if so I'd appreciate a link. This was a confusing rabbit hole to go down, compared to the ease of the Core integration. ---
**Author:** ![GregKalapos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/gregkalapos/32/37205_2.png) [@GregKalapos](https://discuss.elastic.co/u/GregKalapos)\ **Post date:** [May 23, 2020, 11:49am UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/12 "2020-05-23T11:49:54Z")
Hi @erjohnson, I went through the comments here - couple of comments: 1. On the issue on .NET Framework that no span is created for outgoing HTTP requests: 2. About the difference between the Core and the non-Core packages: ---
**Author:** ![erjohnson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/erjohnson/32/68847_2.png) [@erjohnson](https://discuss.elastic.co/u/erjohnson)\ **Post date:** [May 26, 2020, 7:36pm UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/13 "2020-05-26T19:36:39Z")
Thanks, @GregKalapos. After looking at the linked thread and @rajivharris's repo, I think there is another issue. Your solution worked for Elastic.Apm.\* 1.4.0, but after upgrading to 1.5.x, even with the most current release version of the System.Diagnostics.DiagnosticSource package (4.7.1, assembly version 4.0.5.0), the HttpClient spans are again not created. Can you confirm that you can reproduce my issue simply by upgrading one of his sample projects to the 1.5.0 or 1.5.1 Elastic.Apm.\* packages? Updating to the latest _prelease_ version of System.Diagnostics.DiagnosticSource, 5.0.0-preview.4.20251.6, fixes the issue in both 1.5.0 and 1.5.1. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 16, 2020, 3:36pm UTC](https://discuss.elastic.co/t/elk-apm-transaction-not-showing-end-to-end-flow-if-we-have-two-diffent-services/233242/14 "2020-06-16T15:36:52Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.