[JavaOpenTelemetry] Set transaction status to ERROR

Elastic Observability APM
,
1

Our stack/app:

  • Kotlin
  • Quarkus
  • GRPC microservice
  • Opentelemetry

Hi,
I need to set event.outcome to failure for some transactions. Currently it seems that all the transactions are returning success unless the processing ends with exception - hence GRPC error.

I have tried:

LocalRootSpan.current().setStatus(StatusCode.ERROR)

which sadly didn't work.

Does somebody have any idea how to set the status to ERROR?

Thanks :slight_smile:

2

Hi,

If I understand it correctly, the captured grpc transaction state is success, because there is no exception thrown in the processing of it, however, maybe for a business-related reason like invalid user input you need to mark some of those transactions as errors, is that correct ?

Without pushing back on application architecture, I think in this case it would be consistent to propagate the application error to the grpc layer to make the caller aware of the error, but maybe that's not possible here.

Do you have a strong requirement to capture the status on the transaction or would having a span with an error status work as well ?

The reason I'm asking is because if you wrap your processing in a dedicated span, then you should be able to control the state of this span, the only difference is that the transaction will keep a success status, but you will be able to see the failed spans below it.

However, due to the grpc event-driven architecture this might be tricky to implement unless the processing can be wrapped in a single method call.

3

Hi,
sending the GRPC error to the caller isn't really a possibility since we are implementing envoy ext auth interface.

We would like to see some specific transactions in APM as failed. I will give you an example.

The transaction looks like:

envoy -> ext auth ( our app ) -> does some REST calls -> return data to envoy

In the cases where the REST calls fails, we would like to mark these transactions as failed.

The processing is wrapped in single method call and I can set the span to failed. As I mentioned this should be done using

LocalRootSpan.current().setStatus(StatusCode.ERROR)

Im using the LocalRootSpan the same way for labels, since in the APM you can query only the labels which are in the ROOT span of the transaction...

but this didn't mark the transaction as failed.

4

I see, you are calling io.opentelemetry.api.trace.Span#setStatus(io.opentelemetry.api.trace.StatusCode) but the grpc instrumentation likely calls it later and your status is ignored.

However I see that in the grpc instrumentation the status of span is set to ERROR whenever an exception is thrown, if I'm not mistaken this is done here in the opentelemetry java instrumentation: io.opentelemetry.instrumentation.api.instrumenter.DefaultSpanStatusExtractor#extract

So here either throwing an exception or using grpc way to indicate failure by setting the value of io.grpc.Status for the call should probably be better here as the io.grpc.Status status from grpc should be translated to opentelemetry span status.

5

Hey I have reimplemented the solution and it turns out that returning GRPC error status code doesn't make the transaction as event.outcome: failure.

Currently Im returning GPRC RESOURCE EXHAUSTED which end in the transaction labels ->
transaction.result: ResourceExhausted but the outcome is still success.