Apmotel adding resource attributes as labels

Elastic Observability APM
1

I'm hoping someone can help me understand what is happening and how I might fix it.

Setup:
apm-agent-go
open telemetry
Go 1.19

I've been trying to use the otel package for tracing and want to expirement with collecting logs as well. Because of the lack of log support in otel-go, I was forced to experiment with the apm-agent-go package. Configuring it was straight-forward, but I have this problem where the attributes are added as labels and causes APM to lose some "smarts" about the trace.

propagators := []propagation.TextMapPropagator{
		propagation.Baggage{},
		propagation.TraceContext{},
	}
	otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(propagators...))

	os.Setenv("ELASTIC_APM_SERVER_URL", "localhost:8282")
	os.Setenv("ELASTIC_APM_SECRET_TOKEN", "***********")
	os.Setenv("ELASTIC_APM_VERIFY_SERVER_CERT", "true")

	r, err := resource.New(context.Background(),
		resource.WithTelemetrySDK(),
		resource.WithFromEnv(),
		resource.WithProcess(),
		resource.WithAttributes(semconv.ServiceNameKey.String(Service),
			semconv.ServiceVersionKey.String(Version),
			semconv.DeploymentEnvironmentKey.String("dev"),
			attribute.String("service.build", buildVersion)),
	)

	if err != nil {
		return nil, fmt.Errorf("failed to configure default resource: %+v", err)
	}
	apmtrace, _ := apm.NewTracer(Service, Version)
	t, err := apmotel.NewTracerProvider(apmotel.WithResource(r), apmotel.WithAPMTracer(apmtrace))
	if err != nil {
		println(err.Error())
		os.Exit(-1)
	}
	otel.SetTracerProvider(t)

sampled trace:

{
  //...
    "labels": {
      "telemetry_sdk_language": "go",
      "telemetry_sdk_version": "1.17.0",
      "service_name": "test",
      "service_version": "0.0.4",
      "process_executable_name": "server",
      "deployment_environment": "dev",
      "telemetry_sdk_name": "opentelemetry",
      "service_build": "devel",
      "process_runtime_name": "go"
    },
    "observer": {
      "hostname": "*****",
      "type": "apm-server",
      "version": "8.9.1"
    },
    "trace": {
      "id": "52bab839cb81c51536294b92953cee51"
    },
    "@timestamp": "2023-10-12T19:32:03.687Z",
  // ....
    "labels.process_runtime_description": [
      "go version go1.19.10 linux/amd64"
    ],
    "labels.service_name": [
      "test"
    ],
    "host.hostname": [
      "****"
    ],
    "labels.process_runtime_name": [
      "go"
    ]
    // ...
  }
}

What's going on here?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.