# APM service map behavior is different with opentracing
**URL:** https://discuss.elastic.co/t/apm-service-map-behavior-is-different-with-opentracing/272129
**Category:** APM
**Tags:** nodejs, server
**Created:** [May 5, 2021, 2:33am UTC](https://discuss.elastic.co/t/apm-service-map-behavior-is-different-with-opentracing/272129 "2021-05-05T02:33:55Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@jialipassion](https://discuss.elastic.co/u/jialipassion)
#### Post date: [May 5, 2021, 2:33am UTC](https://discuss.elastic.co/t/apm-service-map-behavior-is-different-with-opentracing/272129/1 "2021-05-05T02:33:55Z")
I am using elastic-apm-node agent to create distributed tracing.
Here is my services structures. I have 3 micro services.
producer, kafka, consumer
The call sequence is
producer -\> kafka -\> consumer
I am not using any auto instrument, I am using the `elastic-apm-node-opentracing`, and I am using ipc to communicate between the services.
So here is my code to create the tracing.
- producer
```auto
const agent = require('elastic-apm-node').start({serviceName: 'serviceA'});
const tracer = new Tracer(agent);
const tran = tracer.startSpan('request051');
const span = tracer.startSpan('request5101', {references: [opentracing.followsFrom(tran.context())]});
ipc.connectTo('consumer', () => {
console.log('connected to consumer');
const headers = {};
tracer.inject(tran.context(), opentracing.FORMAT_HTT_HEADERS, headers);
ipc.sendTo('kafka', headers);
ipc.kafka.on('message', () => {
console.log('get response from kafka');
span.finish();
tran.finish();
});
});
```
- kafka
```auto
const agent = require('elastic-apm-node').start({serviceName: 'serviceA'});
const tracer = new Tracer(agent);
ipc.listen((message) => {
const ctx = tracer.extract(opentracing.FORMAT_HTTP_HEADERS, message);
const span = tracer.startSpan('kafka 51`, {childOf: ctx});
ipc.connectTo('consumer', () => {
ipc.consumer.on('message', () => {
span.finish();
ipc.sendTo('producer');
});
});
```
- consumer
```auto
ipc.listen((message) => {
const ctx = tracer.extract(opentracing.FORMAT_HTTP_HEADERS, message);
const span = tracer.startSpan('consumer 51`, {childOf: ctx});
ipc.sendTo('kafka');
span.finish();
});
```
So finally, the distributed tracing timeline looks like this.
Which looks correct, but the `Service Map` has no links.
And I checked the source code of `elastic-apm-node` [apm-agent-nodejs/http-shared.js at master · elastic/apm-agent-nodejs · GitHub](https://github.com/elastic/apm-agent-nodejs/blob/master/lib/instrumentation/http-shared.js#L222) , to display the link of the `Service Map` correctly, the `destionationContext` and the `httpContext` need to be set to the `Span` (There is no documentation about this part, so I am not sure how to set this destination context correctly to do my own instrument). And this `destination context` can be only set to the `Span` object, and a `Span` need to have an `active` transaction, so I need to create a `Transaction` and a `Span` to display the link. So I am using some hacking way to make the link being displayed.
But since I have to create another `transaction` for each `span`, so the timeline looks like this.
You can see in the time line, there is a pair of `transaction` and `span`, they have the same duration, so the `Transaction` is not really useful here, the purpose is just to display the `links` of the Service Map.
To confirm my understanding about the `ServiceMap` and the `timeline` is correct, I am using `opentracing` library with `Jaeger` to display the timeline and the `service dependency`.
And the code is the same with the `elastic-apm-node-opentracing` code I post above.
The final result is:
So in the `Elastic APM`, there are two concepts: `Transaction` and `Span`, and the `Span` can not exist without a `Transaction`, I think this is the key difference, but because of this concept, the behavior different between `APM` and the `opentracing`.
So I want to confirm my understanding is correct or not, and is that possible to be totally compatible with `opentracing` to display service map correctly without additional `Transaction`.
Thank you.
---
### Author:  [@alanstorm](https://discuss.elastic.co/u/alanstorm)
#### Post date: [May 6, 2021, 6:56pm UTC](https://discuss.elastic.co/t/apm-service-map-behavior-is-different-with-opentracing/272129/2 "2021-05-06T18:56:15Z")
Thanks for reporting this @jialipassion -- and, by the way, excellent work in getting as far as you did and communicating that back.
You've caught us in the middle of industry changes here. The short version is that `elastic-apm-node-opentracing` isn't something we're actively supporting -- we're following the industry and putting our efforts [behind OpenTelemetry instead](https://opentelemetry.io/). I'm going to provide some background info on our OpenTelemetry support and then talk about the future of the `elastic-apm-node-opentracing` repo.
With OpenTelemetry there's two options for shipping data to Elastic from your Node.js applications. First, if you have an OpenTelemetry collector receiving data you can export that data from the collector using the [elasticexporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter) from the contrib repository.
Second, if you're running APM Server yourself, we have an experimental feature where APM Server can act-as an OpenTelemetry collector. This will allow you to point the exporters in your Node.js application at APM Server instead of pointing them at an open Telemetry collector. With this option APM Server will honor the `span_kind` attribute any synthesize a transaction for you. You can see the code that does that here: [apm-server/consumer.go at master · elastic/apm-server · GitHub](https://github.com/elastic/apm-server/blob/master/processor/otel/consumer.go#L155-L181)
If you're interested in going to oTel route there's more information here: [OpenTelemetry integration | APM Overview [7.12] | Elastic](https://www.elastic.co/guide/en/apm/get-started/current/open-telemetry-elastic.html)
* * *
So while we've moved on from OpenTracing, the [`apm-agent-nodejs-opentracing` repo is still there](https://github.com/elastic/apm-agent-nodejs-opentracing/issues/18) in order to support folks who find it useful. Updates are limited to community driven PRs at the moment -- we've opened an issue making note of the fact [Service Maps won't work](https://github.com/elastic/apm-agent-nodejs-opentracing/issues/18) -- if anyone reading this has a passion for getting this working we'd love to have that discussion and help figure out what a PR/PRs might look like.
I _think_ that covers it but if you have any follow up questions please don't hesitate to ask. Thanks again for reporting this here!
---
### Author:  [@jialipassion](https://discuss.elastic.co/u/jialipassion)
#### Post date: [May 7, 2021, 12:04am UTC](https://discuss.elastic.co/t/apm-service-map-behavior-is-different-with-opentracing/272129/3 "2021-05-07T00:04:24Z")
@alanstorm, thank you very much for the reply.
1. I am not sure how to make `openTelemetry` to work with `APM`, I am using `openTelemetry` manual instrument, so I assume what I want to do is like the code here [opentelemetry-js/tracing.js at main · open-telemetry/opentelemetry-js · GitHub](https://github.com/open-telemetry/opentelemetry-js/blob/main/examples/collector-exporter-node/tracing.js#L14)
So I can create the `Span` myself and export to the `APM`. But I didn't find the `ElasticSearchAPM` exporter in the `JS` package.
2. Also about the `native collector feature`, [OpenTelemetry integration | APM Overview [7.12] | Elastic](https://www.elastic.co/guide/en/apm/get-started/current/open-telemetry-elastic.html#open-telemetry-elastic-protocol), is that also work with `OpenTelemetry manual instrument`? So is that possible to provide a nodejs sample?
3. And if my understanding is correct, if I am using `OpenTelemetry`, to display the `Service Map` in the `APM` correctly, there will only the `Span` data without the `Transaction` data, is that right?
Thanks again.
---
### Author:  [@alanstorm](https://discuss.elastic.co/u/alanstorm)
#### Post date: [May 10, 2021, 4:57pm UTC](https://discuss.elastic.co/t/apm-service-map-behavior-is-different-with-opentracing/272129/4 "2021-05-10T16:57:27Z")
> I am not sure how to make openTelemetry to work ... I didn't find the ElasticSearchAPM exporter
We hear that -- OpenTelemetry and exporters can be a little tricky to get your head around at first. I'm not exactly sure which part you're having trouble with -- let me know if this gets you pointed in the right direction.
An exporter allows you to get information _out_ of a system that's instrumented with OpenTelemetry. Whether it's manually or automatically instrumented shouldn't matter. If you're generating spans with OpenTelemetry those spans will be handed off to an exporter. You can test this by using the OpenTelemetry `ConsoleSpanExporter`, which "exports" spans by printing them to the console. [This (unaffiliated with Elastic) tutorial](https://alanstorm.com/tracing-nodejs-services-with-open-telemetry/) covers using the `ConsoleSpanExporter` with a simple express based service.
In a real world system an exporter won't send spans to the console. It will send them to whatever span storage system you're using. Broadly speaking there's two strategies exporters take to send data elsewhere. The first is to have an exporter send that data _directly_ to the span storage system. The second is to have an exporter send that data to an OpenTelemetry collector, and the collector will send that data to the span storage system.
Right now Elastic is opting for strategy two. We don't have an exporter that sends data directly to a span storage system. Instead, you'll configure OpenTelemetry with a [collector exporter](https://www.npmjs.com/package/@opentelemetry/exporter-collector), and configure that exporter with a collector URL. With this configuration in place, your spans will be sent to the OpenTelemetry collector.
In other words, the exporter you'll want to use is the `CollectorTraceExporter` exported by `@opentelemetry/exporter-collector`.
## The Collector
Implicit in all this is that you'll need to be [running an OpenTelemetry collector instance](https://github.com/open-telemetry/opentelemetry-collector) somewhere. A collector exposes an endpoint/url, and you point your `CollectorTraceExporter` at this URL.
There's a choice you'll need to make here. You can run the OpenTelemetry collector instance yourself, or you can have APM Server can act as an OpenTelemetry collector.
If you're having APM Server act as the OpenTelemetry collector, then all you need to do is point your `CollectorTraceExporter` at APM Server. The guide information for this [is here](https://www.elastic.co/guide/en/apm/get-started/current/open-telemetry-elastic.html#open-telemetry-elastic-protocol). The docs are specific to java, but should be relatively straightforward to convert to using with the `@opentelemetry/exporter-collector`/`CollectorTraceExporter`. If you run into issues let us know.
If you're running an OpenTelemetry collector instance yourself, then you need to take additional steps to move data from the OpenTelemetry collector to your span storage system. Those additional steps are to use a _collector exporter_. Yes, _another_ exporter.
A collector exporter moves spans from a collector to your span storage system. This is the [contrib exporter](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/elasticexporter) I mentioned in my previous post. The guide information for doing [this is here](https://www.elastic.co/guide/en/apm/get-started/current/open-telemetry-elastic.html#open-telemetry-elastic-exporter).
## Transaction-less Future
Re:
> if I am using OpenTelemetry, to display the Service Map in the APM correctly, there will only the Span data without the Transaction data, is that right?
That is correct. OpenTelemetry's data model is transactionless -- it uses only spans. The limitation you ran into earlier was that the Node.js Agent requires a transaction to be present in order to start a span. Our old OpenTracing solution is based on the agent, so it shared this limitation. With OpenTelemetry, the Node.js Agent isn't in the mix, so this limitation doesn't exist.
Finally -- apologies for the wall of text. 🙂 OpenTelemetry is am ambitious project. If you have additional questions please don't hesitate to keep this conversation going.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 31, 2021, 12:58pm UTC](https://discuss.elastic.co/t/apm-service-map-behavior-is-different-with-opentracing/272129/5 "2021-05-31T12:58:00Z")
This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.