Not support spring cloud gateway?

Elastic Observability APM
,
1

we have an application using spring cloud gateway,and we find some problems on apm,seems like java agent do not support spring cloud gateway (yet) ? see below for details.

Kibana version:7.17.3

Elasticsearch version:7.17.3

APM Server version:7.17.3

APM Agent language and version:Java 1.33.0

Browser version:Chrome

Original install method (e.g. download page, yum, deb, from source, etc.) and version:docker

Fresh install or upgraded from other version? fresh

Is there anything special in your setup? just simple setup,no addition config

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
we use the following version of frameworks:
JDK version: 11
Spring cloud gateway (SCG) version: 3.1.3
spring webflux version(from SCG): 5.3.20
reactor-netty(from spring webflux): 1.0.19

enhancement to our gateway:

  1. we write filters, very simple filters(rewrite url, call redis etc..), which implents org.springframework.web.server.WebFilter (not scg's gatewayfilter, not good choice i know , for some reason i can't change..);
  2. our gateway send outbound requests, just use scg's own feature, which calls reactor-netty directly(yes,they don't use their own webclient);

after setup apm agent, run some requests to gateway, we don't get expected results:

  1. the transaction is stated from spring webflux dispatchhandler, which means the filters implements from webfilters are not captured; these filters are called from defaultwebfilterchain, which is in spring-web module, followed by reactor-netty, not arrive webflux yet.
  2. gateway outbound requests are missing from apm,no http span. the requests use reactor-netty's httpclient directly, i think that's the cause, the elasic apm not support that framework.

by contrast, opentelemetry's java agent works ok: filters, outbounds are captured, and kibana shows well.
but we like elastic apm agent more, it has jvm metrics out of box, and excellent profile/central config features.

so, is this true that elastic apm not support scg yet, or did i miss something?

Steps to reproduce:

  1. write simple scg applications
  2. setup apm agent
  3. run

Errors in browser console (if relevant): no error logs

Provide logs and/or server output (if relevant): nothing special from output

2

I think your analysis is right. I'll reproduce it and see if there's anything obvious/easy to get something working, but I suspect it's a chunk of work

3

Yes. My immediate impression is that it's not a quick thing. Options for you

  1. You can add simplified instrumentation that works for you using our plugin API
  2. You can instrument a few methods directly using the trace_methods option, eg -Delastic.apm.trace_methods=reactor.netty.http.client.*#send
  3. The agent will correctly handle OpenTelemetry spans, so anything that creates those, like this project which can be included could provide what you want

I'll raise an issue to add instrumentation for the gateway and I'll discuss about the filters with my colleagues

4

thank you for your response. i will try to use these options, and see if it works.

5

Finally I wrote intrumentations using opentelmentry-api, now apm seems work as expected.

I would like to share what i did, comments are welcome:

  1. Instrument org.springframework.http.server.reactive.ReactorHttpHandlerAdapter#apply , create transaction on enter, so the filters implements from spring webfilter are captured;
    However, it conflict with elastic's official spring-webflux plugin, which create another transaction, but it does not have to. So dont enable the experimental flag happens to helped me...

  2. Instrument reactor.netty.http.client.HttpClient$ResponseReceiver#responseConnection ，wrap HttpClient.ResponseReceiver and create span on mapConnect, so the outbounds are captured.

These instruments have not yet fully tested. Can someone take a look on this, especially the spring-webflux plugin make some changes like 'only create transaction while the current context dont have one'? @Jack_Shirazi

6

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.