# Elastic Otel does not set environment

**URL:** <https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207>\
**Category:** APM\
**Tags:** open-telemetry\
**Created:** [October 3, 2024, 4:01pm UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207 "2024-10-03T16:01:44Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![dbalijapalli0264](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbalijapalli0264/32/138058_2.png) [@dbalijapalli0264](https://discuss.elastic.co/u/dbalijapalli0264)\
**Post date:** [October 3, 2024, 4:01pm UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/1 "2024-10-03T16:01:44Z")

</div>

**Kibana version** : 8.15.0

**Elasticsearch version** : 8.15.0

**APM Server version** : 8.15.0

**APM Agent language and version** : Elastic-otel-Java agent ; jar file , Release 1.0.0

**Original install method (e.g. download page, yum, deb, from source, etc.) and version**: elastic-otel-agentextension/1.0.0

**Is there anything special in your setup?** For example, are you using the Logstash or Kafka outputs? Are you using a load balancer in front of the APM Servers? Have you changed index pattern, generated custom templates, changed agent configuration etc. NO

**Description of the problem including expected versus actual behavior. Please include screenshots (if relevant)**:

We don't seethe Environment, namespace, being set though it is given as an attribute in the properties.

```auto
    JAVA_PROPERTIES+=" -Dotel.resource.attributes=deployment.environment=${ENVIRONMENT}"
    JAVA_PROPERTIES+=" -Dotel.resource.attributes=kubernetes.namespace=${KUBE_POD_NAMESPACE}"
    JAVA_PROPERTIES+=" -Dotel.resource.attributes=kubernetes.pod=${KUBE_POD_NAME}"
    JAVA_PROPERTIES+=" -Dotel.exporter.otlp.protocol=http/protobuf"
    JAVA_PROPERTIES+=" -Dotel.instrumentation.jetty.enabled=true"
    JAVA_PROPERTIES+=" -Dotel.instrumentation.apache-httpclient.enabled=true"
    JAVA_PROPERTIES+=" -Dotel.exporter.otlp.endpoint=${ELASTIC_OTEL_AGENT_SERVER_URL}"
    JAVA_PROPERTIES+=" -Dotel.instrumentation.methods.include="\

```

Also is this the right way to include custom methods?  
JAVA\_PROPERTIES+=" -Dotel.instrumentation.methods.include="

1.See the attached stack trace for instrumentation,

 ![Screenshot 2024-10-03 at 11.57.29 AM](https://us1.discourse-cdn.com/elastic/original/3X/0/d/0d2342ede1426763e61f1788a257c56241cfb5ee.png) And  
2. Dont see any label in kibana set for the transaction for environment.

---

<div class="post-metadata">

**Author:** ![dbalijapalli0264](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbalijapalli0264/32/138058_2.png) [@dbalijapalli0264](https://discuss.elastic.co/u/dbalijapalli0264)\
**Post date:** [October 4, 2024, 3:12pm UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/2 "2024-10-04T15:12:36Z")

</div>

Hello good morning,  
Wonder of anyone has an answer to this , otherwise, please let m know if this is a bug that needs to be worked on, so that I could submit it as bug on the elastic-otel-java git repo.

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [October 7, 2024, 7:11am UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/3 "2024-10-07T07:11:38Z")

</div>

Hi,

In your configuration you have set multiple values for `otel.resource.attributes`, so the last one will win.

Here you need to set all the 3 resource attributes in the same value as a comma-separated list.

* * *

For now, the `labels.code_stacktrace` comes from the [span-stacktrace](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/span-stacktrace) module that is enabled in our agent, it does not relates to `otel.instrumentation.methods.include`.

* * *

For capturing custom methods with configuration, I don't know and I need to do a bit of research. I think it's possible but not 100% sure yet, however you can already add a few annotations using the OpenTelemetry API as an alternative if you can modify the application.

---

<div class="post-metadata">

**Author:** ![dbalijapalli0264](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbalijapalli0264/32/138058_2.png) [@dbalijapalli0264](https://discuss.elastic.co/u/dbalijapalli0264)\
**Post date:** [October 7, 2024, 8:55pm UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/4 "2024-10-07T20:55:26Z")

</div>

Thank you, I will comma-seperate all the attributes for "otel.resource.attributes" and let you know.

> For now, the `labels.code_stacktrace` comes from the [span-stacktrace](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/span-stacktrace) module that is enabled in our agent, it does not relates to `otel.instrumentation.methods.include` .  
> So, any reason why i see that stacktrace?

> however you can already add a few annotations using the OpenTelemetry API as an alternative if you can modify the application  
> As of now, we can't modify the application, I can find that out.

---

<div class="post-metadata">

**Author:** ![dbalijapalli0264](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbalijapalli0264/32/138058_2.png) [@dbalijapalli0264](https://discuss.elastic.co/u/dbalijapalli0264)\
**Post date:** [October 8, 2024, 4:52pm UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/5 "2024-10-08T16:52:23Z")

</div>

> [@Sylvain\_Juge](#):
>
> In your configuration you have set multiple values for `otel.resource.attributes`, so the last one will win.

That's correct, and I am now able to set different labels and the service.environment. Thank you for the good suggestion.

Regarding  
For capturing custom methods with configuration, I don't know and I need to do a bit of research.

> I cannot modify the code, is there any way this property `otel.instrumentation.methods.include.` be added to the list of the Java properties, for elastic\_otel? As you are already providing this in the elastic apm\_agent java properties. That would be a great help.

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [October 8, 2024, 8:13pm UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/6 "2024-10-08T20:13:27Z")

</div>

When using the elastic distribution of opentelemetry (EDOT), you get the same implementation as the original opentelemetry agent plus few extras, so it should work exactly the same.

So here you should be able to use [`otel.instrumentation.methods.include`](https://opentelemetry.io/docs/zero-code/java/agent/annotations/#creating-spans-around-methods-with-otelinstrumentationmethodsinclude) to capture a custom method without modifying code, however the syntax is not the same as the one used with Elastic APM agent.

---

<div class="post-metadata">

**Author:** ![dbalijapalli0264](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dbalijapalli0264/32/138058_2.png) [@dbalijapalli0264](https://discuss.elastic.co/u/dbalijapalli0264)\
**Post date:** [October 11, 2024, 4:29pm UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/7 "2024-10-11T16:29:48Z")

</div>

> [@Sylvain\_Juge](#):
>
> elastic distribution of opentelemetry (EDOT), you get the same implementation as the original opentelemetry agent plus few extras, so it should work exactly the same.

That helped, I am now able to instrument custom methods.

COuld oyu please help me with understanding what this stack trace is about and how to get rid of it?

```auto
labels.code_stacktrace
at io.opentelemetry.javaagent.shaded.instrumentation.api.instrumenter.Instrumenter.doEnd(Instrumenter.java:263) at io.opentelemetry.javaagent.shaded.instrumentation.api.instrumenter.Instrumenter.end(Instrumenter.java:150) at io.opentelemetry.javaagent.instrumentation.servlet.ServletHelper.end(ServletHelper.java:62) at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:124) at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) at org.eclipse.jetty.server.session

```

---

<div class="post-metadata">

**Author:** ![Sylvain\_Juge](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sylvain_juge/32/55521_2.png) [@Sylvain\_Juge](https://discuss.elastic.co/u/Sylvain_Juge)\
**Post date:** [October 14, 2024, 7:48am UTC](https://discuss.elastic.co/t/elastic-otel-does-not-set-environment/368207/8 "2024-10-14T07:48:53Z")

</div>

Hi,

As written [in feature documentation here](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/span-stacktrace) you should set `otel.java.experimental.span-stacktrace.min.duration=-1` to disable this feature.

It is not a "real" stacktrace in the sense that it does not comes from an application exception, it's just a way for the agent to capture the calling context of the span.
