We have springboot application which uses micrometer tracing bridge otel library to get traces and span. This is working as expected with plain springboot application. When adding elastic apm Java agent trace id, span id is getting disappeared from the logs and not getting appended. Please suggest and let us know if further details
I'm assuming you mean that without the Elastic APM agent, you are seeing trace-IDs and span-IDs in your logs from micrometer, correct?
What you are experiencing is likely the fact that those are overidden from the agent due to its built-in log-correlation feature.
You can try disabling this by setting disable_instrumentations to log-correlation. However, in general I would advice against using micrometer-tracing and the elastic APM agent together, because they are not compatible: e.g. you'll get two distinct traces (one from elastic, one from micrometer) for the same execution. Do you have any reasons for trying to combine those?
Thank you so much for quick reply.
We have ask to use micrometer tracing bridge otel library.
Can you please explain more how this is getting overridden by elastic log correlation feature?
As you said if Elastic apm agent is contradicting with micrometer otel then do you suggest opentelemetry agent instead elastic apm?
Can you please explain more how this is getting overridden by elastic log correlation feature?
They both try to set the trace.id and span.id in the logging MDC. Likely because there is no span active from the perspective of the elastic APM agent, it clears out those values.
As you said if Elastic apm agent is contradicting with micrometer otel then do you suggest opentelemetry agent instead elastic apm?
No, because the opentelemetry agent also doesn't support micrometer tracing, only metrics are "bridged". So you'll have the same problems tehre. You either need to stick with just micrometer tracing or ditch it in favor of the elastic APM Agent or the opentelemetry agent.
Sorry for replying late as I was sick. I have tried disabling instrumentation for log correlation but tit didn’t work. Thank you once again, I have noted all your suggestion. Please let us know if you get any breakthrough on this