APM service map showing less data in more time

[multimaster]

Hi,

We have APM set up for a series of Java services and we have found a rather curious behavior.

For some timespan, the service map of a particular service shows some relations with other monitorized services, this is OK:

If we change the timespan to make it wider, but also containing the previous timespan, the service map show less relations:

¿Why the original relations are missing? ¿Does the service map have a limit in how many transactions uses to be built? ¿Does it use the most common transactions or the most repeated?

Kibana version: 7.14.1

Elasticsearch version: 7.14.1

APM Server version: 7.14.1

APM Agent language and version: Java 1.x

Greetings

[dgieselaar (Dario Gieselaar)]

Hi @multimaster! The service map uses a sampling strategy to detect connections. Selecting a larger time range might lead to some connections being missed due to the fact we cannot inspect all traces one-by-one. However, I'm not sure if that is the case here, to be honest. It might be something else.

In both cases, are you looking at the global service map? And is there anything peculiar about the service that is displayed in both cases?

[multimaster]

Then it must be the sampling strategy. Because the throughput was much higher in the prior hours (11-13:30) and this throughput was likely coming off someplace else which is not those services. So then, the service map misses the transactions coming from the monitorized services due to low volume compared to all the traffic. Starting 13:30, we can see it as the other traffic fades away.

We were seeing this specific service service map. In the others service's service map as well as the global one, we can see all the relations OK.

This sampling strategy, how does it work? Is it completely random? I mean, it might lead to different results if you put another request for the same timespan? What is the chance to miss some transactions?

Thanks for the quick answer