Java APM tag external http request made with reactive webclient

[mrodal]

Kibana version: 8.6

Elasticsearch version: 8.6

APM Server version: 8.6

APM Agent language and version: 1.35.0

Browser version: -

Original install method (e.g. download page, yum, deb, from source, etc.) and version: Download page

Fresh install or upgraded from other version?: Fresh

Is there anything special in your setup? No

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
Im trying to tag external http requests made with webclient. Based on this issue, im trying to use a webclient filter to get the current span and add a tag, but the current span I get is the parent one, not the external request span.

Looks like the external span is not started when the filter is executed, the filter executes before this method.

I believe the span should be started before in the call stack to allow this customization, here perhaps.

[carly.richmond]

Hi mrodal,

Thanks for the details. Had a chat with the team and they are taking a look. We'll keep you posted!

[Jonas_Kunz]

Hi @mrodal,
Could you please share a snippet of how you use the spring webclient?
Unfortunately the reactive nature of the webclient makes it very hard to propagate the context to every possible location.

I tried locally, the span should be available within the lambda passed to the exchange methods:

client.get().uri(uri).exchangeToFlux(response -> {
                            Span span = ElasticApm.currentSpan(); //should be the correct span here
                            return response.bodyToFlux(String.class);
                        }
                    )
                    .blockLast();

[mrodal]

Hi @Jonas_Kunz ,
The idea wat to apply a similar approach to the interceptors with restTemplate (which work as expected) using webclient filter functions to apply the span transformations to all subsequent requests.

I had not tried your approach getting the span on the exchange function, it would require to add the label on each request, but it could work as last resort. The issue is that I also get the parent transaction there, heres my code:

WebClient client = WebClient.builder()
        .filter((clientRequest, next) -> {
            ElasticApm.currentSpan().setLabel("label1", "from weblcient filter");
            return next.exchange(clientRequest);
        })
        .build();

String response = client.post()
        .uri("https://httpbin.org/post")
        .body(BodyInserters.fromValue("test body"))
        .exchangeToMono(response -> {
                    Span span = ElasticApm.currentSpan().setLabel("label2", "from exchangeToMono");
                    return response.bodyToMono(String.class);
                }
        )
        .block();

I apply a label on the filter function and another one on exchangeToMono, but I get both applied to the parent transaction:

[Jonas_Kunz]

I was able to reproduce the behaviour you described. I think I initially missed this because I had the webflux instrumentation disabled.

Anyway, I think I found a solution which should work in your case:

    private static class SpanAwareExchangeFunction implements ExchangeFunction {

        private final ExchangeFunction delegate;

        private final Consumer<Span> actions;

        public SpanAwareExchangeFunction(ExchangeFunction delegate, Consumer<Span> actions) {
            this.delegate = delegate;
            this.actions = actions;
        }

        @Override
        public Mono<ClientResponse> exchange(ClientRequest request) {
            actions.accept(ElasticApm.currentSpan());
            return delegate.exchange(request);
        }
    }

    private static void doWebclientRequest() {
        WebClient client = WebClient.builder()
                .filter((clientRequest, next) ->
                    new SpanAwareExchangeFunction(next, span -> {
                        span.setLabel("label1", "from webclient filter");
                    }).exchange(clientRequest)
                )
                .build();

        String response = client.get()
                .uri("https://httpbin.org/get")
                .exchangeToMono(response -> response.bodyToMono(String.class))
                .block();
    }

Why does this work? Our webclient instrumentation targets classes implementing the spring ExchangeFunction interface which also have ExchangeFunction in their name. This means in the code above the span is already started within SpanAwareExchangeFunction and therefore is accessible to you. It is important that SpanAwareExchangeFunction is not a lambda, so that the name contains "ExchangeFunction", because otherwise it will not be instrumented.