Logging integration with OpenTelemetry

[fbutter (Florian Butter)]

OpenTelemetry and W3 defines span and trace id but no transaction id.

How to do the log integration without the transaction id?

Kibana version: (Cloud) 8.5.3

Elasticsearch version: (Cloud) 8.5.3

APM Server version: (Cloud) 8.5.3

APM Agent language and version: opentelemetry-javaagent v1.31.0

[kyungeunni (Kyungeun Kim)]

Hi @fbutter,

if the log is sent from an OpenTelemetry agent to the APM Server, APM Server will translate the root span or certain kinds of spans as a transaction when receiving otel data so that the logs will contain the transaction ID which can be the same as the root span ID.

Here's how to set up the exporter for the logging: OpenTelemetry native support | APM User Guide [8.10] | Elastic

[fbutter (Florian Butter)]

Hi @kyungeunni , thank you.

I'm sorry, my question was misleading.

We are using OpenTelemetry for the sending metrics and traces to the monitoring backend. The logs are sent via Filebeat.

In detail, we use the span and trace id from the OpenTelemetry agent and build (together with other log relevant information) a structured log (JSON) that is sent to the monitoring backend.

[fbutter (Florian Butter)]

Is the OpenTelemetry logging integration with Elastic Observability production-ready although the Events API is still experimental?

[fbutter (Florian Butter)]

Found the info here: Its in preview state Limitations | APM User Guide [8.10] | Elastic

[kyungeunni (Kyungeun Kim)]

Hi Florian,

So sorry for getting back to you so late.
Yes OpenTelemetry logging integration with Elastic Observability is in preview state indeed.

We are using OpenTelemetry for the sending metrics and traces to the monitoring backend. The logs are sent via Filebeat.
In detail, we use the span and trace id from the OpenTelemetry agent and build (together with other log relevant information) a structured log (JSON) that is sent to the monitoring backend.

We don't have support for that yet, but you might want to try duplicating every id in the log and set it as both a transaction and span ID. I have not tried it on my side so it might not work and the UI could break, that's what we can suggest right now. Apologies for not being super helpful. I will deliver this case to our product team.

Thanks,
Kyungeun