Tail-based sampling

Elastic Observability APM
,
1

Kibana version: 8.7.0

Elasticsearch version: 8.7.0

APM Agent language and version: .NET

Description of the problem including expected versus actual behavior. Please include screenshots (if relevant):
Hello, I've configured tail-based sampling on server using this configuration :

- sample_rate: 1 
  trace.outcome: "failure"
- sample_rate: .01

But when I look in Kibana, I always see all traces including transactions and spans data for succesful traces.
What is a proper configuration client/server side to have only 1% of sampled traces ?
I use the .NET Elastic API to start and close transactions. I hope sampling is not based on "IsSampled" property because it's always "true" and it cannot be determined client side if sampling is configured in tail-based mode on server side.

Thanks in advance,

1 Like
2

With tail based sampling, whether a trace is sampled or not, is decided based on the root transaction. The root transaction is the very first transaction in a distributed trace. You can identify root transactions as transactions who do not have a parent.id set.
The sampling rate is defined for trace.outcome , which means event.outcome of the root transaction. With the configuration that you shared, all traces (transactions+spans) where the root transaction has a event.outcome: failure are going to be 100% sampled, independent of the event.outcome of the other transactions in this trace. And traces (transactions+spans) where the root transaction has any other value for event.outcome, are sampled with 1%.

Can you please verify the sampling again with looking at the root transactions?

3

Hello,

I've made more than 20 calls where root transactions has event.outcome equal to success and all traces are visible in Kibana including transactions and spans data.
We are using an old version of NuGet Package Elastic.Apm : version 1.8.1, maybe this version doesn't support tail-based sampling ?

Thank you for your help,

4

I've got news. It's eventually working today ! I don't know why the situation changed. Maybe applications rebooting solved the problem.
Sorry for the disturbance.

Thank you for your time,

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.