ELK APM and Jaeger Integration add Kubernetes metadata and system data to transactions/spans

[zvazquez (Zareh Vazquez)]

Hi,

We are working on the integration of Jaeger with ELK APM running on Kubernetes, we noticed that some of the metadata and actual data related with Kubernetes and APM specific is not available, and we could not find a way to add it, by example:

• ELASTIC_APM_ENVIRONMENT
• kubernetes.pod.uid
• kubernetes.pod.name
• system metrics
• etc.

Is there any way we can enrich the data either using the jaerger libraries, opentracing libraries or maybe using parameters in the agent? By example we tried --jaeger.tags, but when used this stopped the span to flow from the agent into the APM server.

Thanks in advance for your help,

Zareh

[axw (Andrew Wilkins)]

@zvazquez

We are working on the integration of Jaeger with ELK APM running on Kubernetes, we noticed that some of the metadata and actual data related with Kubernetes and APM specific is not available, and we could not find a way to add it, by example:

• ELASTIC_APM_ENVIRONMENT
• kubernetes.pod.uid
• kubernetes.pod.name

Right now there is no way to define these, as they do not exist in Jaeger/OpenTracing semantic conventions.

We will be adding support for OpenTelemetry Protocol to APM Server (elastic/apm-server#4503), and this will involve translating OpenTelemetry semantic conventions to the Elastic APM model. When we do that, we'll consider whether it's safe and sensible to translate these conventions for Jaeger as well.

• system metrics
• etc.

This one is quite different. Jaeger does not send system metrics.

If you need metrics integrated in the APM app in Kibana, you would be best off using the Elastic APM agents. Otherwise you will need to supplement, e.g. using Metricbeat. Just bear in mind that these metrics will instead show up in the Metrics (not APM) app in Kibana; they are not as tightly integrated.