Setting Environment for Traces Collected by OpenTelemetry Collector

Kibana version: 7.17.2

Elasticsearch version: 7.17.2

APM Server version: 7.17.2

APM Agent language and version: Open Telemetry Collector

Browser version: N/A

Original install method (e.g. download page, yum, deb, from source, etc.) and version: ECK

Fresh install or upgraded from other version? N/A

Is there anything special in your setup? For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc.

  • Runs in K8s, but not super relevant

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):

Using the Open Telemetry Collector (For traces generated by something like istio) use the attributes processor to upsert a service.environment attribute. The APM server will map that to labels.service_environment, rather than service.environment.

Not sure how get service.environment, especially in light of: Set default `service.environment` · Issue #4468 · elastic/apm-server · GitHub which is going to remove the undefined environment option.

Steps to reproduce:
1.
2.
3.

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

In DotNet I had to specifically add attributes on the Resource builder.

With out those values the Metrics or Traces dashboards functionality is limited.

I don't know how exactly you do this for your situation but I hope its a direction that can help you solve it

  • deploymnet.environmnet actually gets transformed to to service.environment (in the Exporter code for DotNet) - I didn't have time to change it to the OTLP standard name as you suggested

I appreciate the dotnet implementation, the problem is/was that I can't modify these services as they are istio services, not direct code. They are just another network hop. But seeing as they are the same name in each env it made it difficult to separate.

APM Processing OTEL Collector Attributes to Labels Rather than Attributes · Issue #8074 · elastic/apm-server · GitHub is the issue I filed, and got a resolution to.