Dropped unsampled transactions in >= 8.x poses a problem


If I'm not mistaken, since version 8.0 we cannot collect unsampled transactions. (PR:
Remove `apm-server.sampling.keep_unsampled` config by axw · Pull Request #6514 · elastic/apm-server · GitHub). No option can turn that on anymore.

I haven't found any documentation about that change, but it seems it has posed a serious limitation for setups with sample rate below 1.0. Without sampling all our transactions we don't anymore have info about the exact count of our backend requests, nor sum of their duration, nor time of their occurrences.

We see that info only for percentage of transactions defined in sample rate.

Is there any way to go around it or the only way since 8.x is to turn 100% sampling rate?

Thanks for any help,

Hi @Pawel_Bakiewicz

All the information that was captured in unsampled transactions is now captured in the form of transaction metrics, in a much more efficient way. These metrics are used by the APM app to populate all the different graphs, so there shouldn't be much of a difference compared with how the graphs and data looked in pre 8.0 with unsampled transactions included. Does it look different for you?

Unfortunately it does.

Your link points to legacy APM, there is no aggregation option in new input settings.

Most importantly, I'd like to know total duration for my backend transactions. Before 8.x I could simply do sum of transaction.duration.us . How can I achieve that without info about duration of unsampled transactions?

Would really appreciate any assistance since I'm thinking about moving back to 7.x :expressionless:

Hey there @Pawel_Bakiewicz!

Unfortunately, even if the Python agent had a config value for sending unsampled transactions, the APM Server would drop them. Because we can use upscaling from the samples to fill out all the metrics, being able to avoid storing all of that extra data is a huge win for most users.

Out of curiosity, why is the total duration important to you? Most users care more about the averages and outliers, not the total.

This topic was automatically closed 20 days after the last reply. New replies are no longer allowed.