Hello everyone,
In our project we have a server which works as a router (i.e. routers calls from one server to other appropriate servers).
We have APIs as action API and callback APIs. This means, server1 would call router, which would call server2. After sometime the server2 will call router with the result of the request which router will forward to server1 back.
In short
server 1 => action API on router => server 2
server 2 => callback API on router => server 1
We wanted to figure out if it is possible to capture how many calls are going through this whole cycle and how many of them might be failing?
Please let us know if it is possible to fulfil this requirement using APM. We have tried starting a custom transaction using co.elastic.apm:apm-agent-api, but there seems to be no way to find an existing transaction and stop it in another API call.
Note: We are using multi replica containers, so we might not receive the action and callback on the same pod.
In your case, you should look for Web frameworks, HTTP clients and RPC frameworks. Only if you are using frameworks/libraries that are not supported for cross-server communication, you should have to use the API.
Thus, the first question is what framework/library you are using to implement those server endpoints and to perform calls to other servers ?
Thanks for your reply. Our servers are running using Spring Webflux, which uses Netty under the hood. From this issue, I understand that spring webflux is not supported right now.
However, If I were to go ahead doing it programatically, what approach would you suggest. I have tried including apm-agent-api and taking below steps:
Start a transaction in the first api call. Add a label with a request-id
Find the transaction with the given request-id(same as previous one) in the subsequent api call and end it. However we are not able to find a way to get an existing transaction.
Please let us know, what would be the right approach to solve this.
In that case, the easiest way to do that is to use the notion of "context" that is provided by Webflux to store the transaction/scope, for example using the strategy described here : https://ndportmann.com/passing-context-with-spring-webflux/ (I've only skimmed over the content, I haven't tried it in practice).
As long as the Transaction object is not ended, it won't be sent and recycled, thus you have to make sure to not leak a reference in the context.
We are already using for webflux context for few other requirements (like implementing MDC traceid logging). However in this case the scenario is a bit different. I need to associate a single transaction between two API calls on the same service(maybe be a different pod).
On /action I start a transaction (with a unique label)
On /callback I find the transaction(using the label) started as part of action call and I end that.
The problem is I don't see a way to find the transaction from APM APIs. I cannot have it in memory because the request can go to different pods of the same service.
Just bumping this up, for a clarification if our requirement is possible at all to solve using APM. If not please let us know, we will try to find other means to do it.
If I understand correctly, your use case seems to be about grouping together two distinct transactions that are created for /action and /callback into a single trace, right ?
Does those requests are coming from a browser ? If yes, I think that using the RUM agent might allow to create a single trace and propagate the headers to group those two requests into the same trace.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.