Celery Transaction not stored in the correct service

[francois.tarlier]

Hi,

We are monitoring several celery services using the elasticapm python agent like follow :

import elasticapm
from elasticapm import Client, instrument, events
from elasticapm.contrib.celery import register_exception_tracking, register_instrumentation
instrument()
apm_client = Client(service_name=app_name)
register_instrumentation(apm_client)
register_exception_tracking(apm_client)

we ONLY do celery automatic instrumentation, no custom code monitoring.

For some unknow reasons, some transactions are not stored under the correct service name.
Lets assume we have ServiceA, ServiceB, ServiceC each reporting celery instrumentations + Metrics via psutils.
ServiceA, ServiceB and ServiceC are showing up in my services page. But transaction on ServiceA and ServiceB are empty, only metrics are reported. ServiceA is triggered by Celery Beat Schedule every 5 sec. Then ServiceA calls ServiceB via delay().
Then if I open ServiceC, I can find some of ServiceA, ServiceB transactions, as of some ServiceC.

elasticapm : 5.10.1
elasticsearch/kibana : 7.9.2

[beniwohli (Benjamin Wohlwend @ APM)]

Hi @francois.tarlier

that sounds weird. Can you tell me a bit about the different services and how they are deployed? Are they all running in different processes? And do they share the same code base or are they completely different (the fact that you use delay to call ServiceB from ServiceA indicates that they share the same code base)? And can you confirm via e.g. logging that the actual tasks run in the services that you expect them to be running in?